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ABSTRACT 


X 

This  research  evaluates  the  operational  efficiency  of  the 
in-house  real  property  maintenance  activity  of  the  Air 
Training  Command  using  a  methodology  called  Data  Envelopment 
Analysis  (DBA).  Evaluations  are  undertaken  in  a  variety  of 
ways  —  reviewing  annual  data;  checking  for  trends,  stability, 
and  seasonal  behavior  using  window  type  analyses;  and 
accomplishing  a  joint  analysis  with  Tactical  Air  Command  data. 
Results  include  identifying  sources  and  amounts  of 
inefficiencies  for  each  base,  command-wide  trends,  and  special 
operational  characteristics  of  different  bases. 
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CHAPTER  1 


INTRODUCTION  AND  BACKGROUND 

SECTION  I  -  INTRODUCTION 

The  purpose  of  this  research  was  to  evaluate  or  measure  the 
operational  efficiency  of  the  in-house  real  property  maintenance 
activity  of  the  Air  Training  Command.  Here  efficiency  is  defined 
as  the  ratio  of  benefits  achieved  (outputs)  to  resources  used 
(inputs) . 1 

Often,  as  in  commercial  accounting,  efficiency  is  measured 
by  comparing  an  actually  attained  output  to  a  standard  or 
predetermined  output.  In  engineering,  outputs  and  inputs  are 
customarily  measured  in  terms  of  energy,  so  that  a  natural  unit 
is  thereby  provided.  Also  the  law  of  conservation  of  energy 
requires  that  the  energy  produced  (output)  must  not  exceed  the 
energy  consumed  (input).  Since  all  units  of  measurement  are  the 
same,  a  dimensionless  ratio  results  with  (K  Efficiency  O  in  this 
ratio  form. 

» 

Unfortunately  these  concepts  are  not  normally  applicable  to 
Air  Force  organizations.  If  we  were  able  to  specify  a  single 
output  like  the  maximum  achievable  production  of  flying  hours  of 
a  wing,  given  specified  levels  of  resources,  then  wing  efficiency 
could  be  determined  by  comparing  the  actual  production  of  flying 
hours  to  the  predetermined  maximum  achievable  flying  hour 
production.  No  production  function  has  been  developed  which  can 
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forecast  the  maximum  number  of  flying  hours  achievable  given  the 
multitude  of  resource  combinations  and  environmental  conditions. 
Thus,  Air  Force  organizations  must  rely  on  relative  measures  of 
efficiency  from  empirically  based  comparisons  of  input  and  output 
measures. 

In  economics,  efficiency  is  usually  assumed  to  have  been 
achieved  by  the  force  of  market  competition.  In  our  case,  we  are 
dealing  with  not-for-profit  military  operations  where  the 
assumption  of  perfect  competition  is  not  tenable.  Hence,  for 
this  research  we  cannot  make  such  an  assumption  and  are  therefore 
forced  to  turn  to  an  efficiency  measurement  methodology  called 
Data  Envelopment  Analysis  (DEA)  to  ascertain  whether  technical 
efficiency  has  been  achieved. 

Our  research  will  proceed  as  follows.  In  the  remainder  of 
this  chapter  we  will  briefly  describe  DEA  and  then  identify  and 
define  the  different  input  and  output  measures  used  in  this 
research.  Chapter  2  reports  and  interprets  the  results  of 
applying  DEA  to  the  Air  Training  Commands  in-house  real  property 
maintenance  data.  Our  summary  and  conclusions  are  contained  in 
Chapter  3* 


SECTION  II  -  DATA  ENVELOPMENT  ANALYSIS 

ke  now  turn  to  a  description  of  Data  Envelopment  Analysis 
(DEA)  as  the  method  we  will  use  to  approach  our  research. 
Charnes,  Cooper,  and  Rhodes  (CCR)  [6]  and  [7]  developed  DEA  to 
measure  and  evaluate  the  relative  efficiency  of  operations  in 
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not-for-profit  programs.  In  order  to  keep  this  paper  from 
becoming  too  technical,  we  only  summarize  the  DEA  model  and  its 
properties  and  characteristics. 

Fractional  Model 


The  following  DEA  model  and  its  associated  extremal 
principals  extend  the  normal  single  output  to  single  input 
efficiency  definitions  employed  in  the  natural  sciences  to  the 
multiple  output  and  multiple  input  case  we  need. 


Maximize: 


ur>rO 
l?,  yix10 


(1) 


Subject  to: 

1  - 

ur*  vi  -  €  >  0 


where  the  terms  represent: 

hg  s  The  measure  of  efficiency  for  decision  making 
unit  (DMU)^  "0W,  the  member  of  the  set  of 
js  1 , . . .  ,n  DMUs  that  is  to  be  rated  relative  to  the 
others.  The  ratio  on  which  hg  depends  is 
represented  in  the  functional  for  optimization  as 
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well  as  in  the  constraints.  This  DMU  preserves 
its  original  subscript  identification  in  the 
constraints  but  is  distinguished  by  a  "0" 
subscript  in  the  functional. 

ur  s  The  variable  for  each  type  of  output  "r" ,  which 

will  be  optimally  determined  by  the  solution  of 

■j 

the  model  and  assigned  as  a  weight*'  to  the 
observed  output  value,  yp . 

v^  s  The  variable  for  each  type  of  input  "i" ,  which 
will  be  determined  by  the  solution  of  the  model 
and  assigned  as  a  "virtual  multiplier"  to  the 
observed  input  value,  . 

yro  =  The  known  amount  of  output  "r"  produced  by  DMU  "0" 
during  the  period  of  observation. 

xio  a  The  known  amount  of  input  "i"  used  by  DMU  "0" 
during  the  period  of  observation. 

yrj  a  The  known  amount  of  output  "r"  produced  by  DMU  "j" 
during  the  evaluation  period. 

xiJ  *  The  known  a®ount  of  input  "i"  used  by  DMU  "j" 
during  the  period  of  observation. 


€  >0  s  A  snail  "non-Archinedean"  constant. 


All  of  the  organizations  are  assumed  to  have  common  inputs 
and  outputs  in  positive  amounts.  Execution  of  the  model  requires 
repeated  computations  which,  in  principle,  must  be  done  for  each 
DMU  in  the  universe  of  organizations  under  evaluation.  In  each 
case,  the  efficiency  of  each  DMU  is  calculated  in  relation  to  all 
other  DMUs. 

The  resulting  efficiency  value,  hg,  does  not  depend  on  the 
units  of  measure  in  which  the  inputs  and  outputs  are  stated. 

That  is,  if  any  input  and  output  is  measured  in  different  units 
then  the  value  of  hg  will  not  alter  provided  this  same  change  is 
made  in  the  units  of  measure  for  all  DMUs. 

Evidently  the  maximum  value  of  hQ  is  unity  since  the 
constraints  require  hg  _<1.  Indeed  if  hg  <1,  then  some  convex 
combination  of  other  DMUs  could  have  done  better  and  DMU  "0"  is 
not  efficient.  Conversely,  DMU  ”0”  is  efficient  if  and  only  if 
hg  =1. 

We  can  relate  this  to  the  concept  of  Pareto  optimality  by 
saying  that  a  decision  making  unit  is  efficient  if  and  only  if  it 
is  not  possible  to  augment  any  output  without  either  (a) 
decreasing  some  other  outputs  or  (b)  augmenting  some  inputs. 
Alternatively,  it  is  inefficient  if  some  input  can  be  decremented 
without  worsening  any  output  or  without  increasing  some  other 
input.  This  does  not  preclude  making  tradeoffs  after  the 
”e"°iciency  frontier”  is  attained  but  it  does  require  setting 
them  aside  until  after  this  frontier  is  identified. 


Reference  to  Figure  1  will  help  to  show  what  is  involved. 
The  solid  line  connecting  points  A,  B,  and  C  represent  a  section 
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of  the  unit  isoquant,  i.e.,  the  level  of  the  production  surface, 
for  one  unit  of  a  single  output.  For  simplicity  we  restrict 
ourselves  to  the  case  of  one  output  (produced  at  unit  level)  and 
two  inputs,  and  Xg*  The  x^  and  X2  coordinates  of  points  A,  B, 
C,  D,  E,  and  F  represent  observed  inputs  used  to  produce  the  one 
unit  of  output  attained  by  each  of  the  six  DMUs  associated  with 
these  points. 


Figure  1:  PEA  Efficiency  In  The  Single  Output  -  Two  Input  Case 

Both  D  and  E  are  inefficient  since  they  are  dominated  by  b* 

11 

and  E',  respectively.  The  latter  are  not  actually  observed 

values  but  are  obtained  as  convex  combinations  of  A  and  b  and 

B  and  C,  respectively,  which  represent  elements  of  the  efficient 

* 

frontier  production  possibility  set.  In  fact,  the  values  of  hQ 

for  points  D  and  E  correspond  to  the  ratios  of  the  ray  segments 

d (0-D • ) /d( 0-D )  and  d(0-E 1 )/d(0-E)  which  are  clearly  less  than 
5 

unity. 
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The  points  A,B,  and  C  from  which  these  convex  combinations 
are  obtained  are  all  efficient  and  form  an  efficiency  frontier. 
There  is  no  point  that  can  be  generated  from  convex  combinations 
of  members  of  the  production  possibility  set  that  will  dominate 
them.  Conversely,  any  movement  along  this  frontier  requires 
tradeoffs  between  x^  and  Xg  in  order  to  stay  on  the  frontier. 

The  DEA  model  evidently  provides  only  relative  evaluations 
by  creating  an  "efficient  frontier"  such  as  the  one  depicted  in 
Figure  1  that  is  generated  from  actual  observations.  It  is 
relative  in  the  sense  that  the  efficiency  rating  depends  on  the 
DMUs  used.  Although  DEA  does  depend  on  the  DMUs  used,  it  does 
not  depend  on  prior  theoretical  knowledge  or  explicit  assumptions 
about  the  value  of  the  production  process  as  in  the  model 
specifications  used  in  statistical  regression  (and  like) 
approaches . 

The  ur  and  v^^  valuer  described  in  model  (1)  may  be 
considered  weights,  but  to  avoid  confusion  with  normal  uses  of  a 
priori  weights,  we  refer  to  the  u^  and  v^  variables  as 
transformation  ratios.  This  name  refers  to  the  fact  that  they 
transform  real  inputs  (xiQ)  to  a  "virtual"  input  (XQ)  and  real 
outputs  (yr0)  to  a  "virtual"  output  (YQ).  In  this  way  the  DEA 
approach  reduces  the  multiple  outputs  and  the  multiple  inputs  to 
a  single  scalar  measure.  Finally,  the  ur  and  Vj^  choices  made  by 
the  DEA  model  are  optimal  in  that  the  mathematical  procedure 
places  the  DHU  that  is  being  evaluated  "in  the  best  possible 
light"--  in  the  sense  that  no  other  u  and  v4  values  can  give  a 
more  favorable  efficiency  ratio  from  this  set  of  data. 


ataa 
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Reduction  To  Linear  Programming  Form 


The  model  previously  presented  is  a  non-linear  programming 
problem.  It  is,  in  fact,  a  fractional  programming  problem  with  a 
linear  fractional  objective  and  linear  fractional  constraints. 

As  such  it  is  both  nonlinear  and  nonconvex.  However,  Charnes, 
Cooper,  and  Rhodes  have  shown  that  it  in  be  transformed  into  an 
equivalent  linear  programming  problem  by  means  of  the  theory  of 
linear  fractional  programming,  developed  by  Charnes  and  Cooper. 

In  order  to  simplify  matters  we  are  bypassing  the  development  of 
the  linear  programming  form  and  present  it  as  follows: 


Minimize : 


hn  =  0  -  € ( 


t 

£ 

rs/ 


s. 


?s- 
Ui  ■ 


(2) 


Subject  to: 


-  s 


r  *  ^rO'  . s 


j*»  J  r  j  j 

-2  Xi  *x 
1 J 

Xj»  sj,  sj  >  0|  0  unrestricted  in  si^n 


+  0xio  =  0 •  i=1 


i  •  •  •  > 


m 


where 


0  *An  intensity  valuer  or  multiplier  of  the  observed  input  x. 
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S*  s  Output  slack  for  output  "r". 

S"  =  Input  slack  for  input  "i". 

€  =  A  small  positive  valued  non-Archimedian  constant. 


To  enforce  the  non-Archimedean  character  of  g  and  avoid 

possible  troubles  from  using  "small"  real  numbers  we  revert  to 

0 tD 

the  procedure  described  in  Charnes  and  CooperAand  first  minimize 
6  with  the  constraints  shown  in  (2)  remaining  unchanged.  Then  we 
maximize  the  slack  variables  in  the  objective  function  while 
constraining  0  to  the  value  it  already  attained  in  the  first 
stage. 

Model  (2)  is  the  form  which  is  used  in  our  research.  For  a 

unit  to  be  rated  100%  efficient  0*  must  equal  one  and  all  slack 

e  * 

variables,  s^  and  sp  ,  must  equal  zero.  Hence  0*  <1  and/or 

* 

si>0  means  that  the  observable  inputs  were  excessive  and 
efficiency  was  not  achieved. 

Comparison  Sets 

* 

Also,  from  model  (2),  we  have  X.  >  0  as  a  sufficient  but 

J 

1.  L. 

not  necessary  condition  for  the  jtn  DMU  to  be  a  member  of  the 
comparison  (sreference  or  neighborhood)  set  of  the  evaluated 
unit.  Recall  that  the  optimization  employed  in  (1)  ensures  that 
the  efficiency  reference  set  provides  the  "best"  (=higiiest)  hy 
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value  available  for  each  DMU.  Knowing  the  DMUs  from  which  the 
evaluation  was  made  allows  managers  of  inefficient  units  to  check 
with  those  organizations  on  possible  corrective  actions. 

Finally,  a  DMU  rated  100J  efficient  which  does  not  appear  in 
the  efficiency  reference  set  of  other  DMUs  which  are  rated 
Inefficient  is  a  candidate  for  additional  review.  Since  a 
comparison  set  cannot  contain  any  inefficient  DMUs,  failure  to 
appear  in  such  a  set  is  an  indication  that  this  DMU  may  be  a 
"self  evaluator"  and  should  not  be  considered  efficient  without 
further  investigation.  The  DMU  may  be  wholly  efficient  because 
it  has  special  features  distinguishing  it  from  the  others  or  it 
is  possible  that  the  DMU  is  operating  inefficiently. 

SECTION  III  -  SPECIFICATION  OF  DEA  INPUT/OUTPUT  MEASURES 

Specification  of  the  input  and  output  measures  to  be  used  in 
the  DEA  model  was  done  in  conjunction  with  major  air  command 
(MAJCOM)  civil  engineering  officials.  The  specific  measures 
chosen  were  based  on  the  operational  expertise  of  these 
individuals  and  certain  characteristics  that  the  input  and  output 
measures  should  have  in  order  to  take  advantage  of  the 
capabilities  of  the  DEA  model. 

There  are  four  guidelines  pertinent  to  the  selection  of  the 
inputs  and  outputs. 

First,  the  inputs  and  outputs  should  be  comprehensive.  That 
is,  they  should  fully  and  properly  measure  the  in-house  real 
property  maintenance  activity. 
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Second,  there  should  be  some  basis  for  believing  chat  the 
relationship  between  inputs  and  outputs  should  be  such  that  an 
increase  in  an  input  can  reasonably  be  expected  to  increase  one 
or  more  of  the  outputs. 

Third,  all  input  and  output  measures  should  exist  in 
positive  amounts  for  each  DMU. 

Finally,  the  variables  should  be  identifiable  and  defined 
and  controlled  so  that  they  cannot  be  manipulated  in  reports  or 
at  least  the  resulting  data  should  be  reviewed  in  order  to  remove 
these  effects  which  might  otherwise  influence  the  results  of  the 
DEA  model . 

A  number  of  possible  input  and  output  measures  were  reviewed 
and  discussed  for  possible  inclusion  in  the  Data  Envelopment 
Analysis  phase  of  this  research.  The  potentially  large  number  of 
possibilities  was  narrowed  to  ones  which  seem  to  best  fulfill  the 
requirements  of  this  study. 

Eight  outputs  were  considered  which  comprehensively  reflect 
the  accomplishments  of  the  base  civil  engineering  activity  in 
performing  its  in-house  real  property  maintenance  function.  We 
label  and  number  these  outputs  as:  (1)  number  of  completed  job 
orders,  (2)  number  of  completed  work  orders,  (3)  number  of 
completed  recurring  work  actions,  (4)  number  of  delinquent  job 
orders,  (5)  replacement  value  of  structures  and  systems,  (6) 
percent  of  job  orders  completed,  (7)  percent  of  work  orders 
completed,  and  (8)  percent  of  recurring  work  actions 
accomplished . 


— — 


There  are  six  inputs  representing  the  resources  consumed  and 
effort  expended  in  producing  the  outputs.  They  are:  (1)  funds 
available  in  terms  of  supply  and  equipment  funding,  (2)  available 
direct  labor  hours,  (3)  number  of  passenger  carrying  vehicles 
assigned  and  available  for  use,  and  (4)  three  measures  of  the 
amount  of  work  available  for  accomplishment  —  number  of  work 
orders  in  the  civil  engineering  system,  number  of  job  orders  in 
the  civil  engineering  system,  and  number  of  scheduled  recurring 
work  actions. 

The  results  of  our  efforts  to  gather  data  on  these  variables 
was  disappointing,  in  that  an  insufficient  number  of  observations 
were  available  for  a  stable  Data  Envelopment  Analysis  of  more 
than  seven  variables.  Therefore,  we  reduced  the  number  of  actual 
measures  used  to  Outputs  1,2, 3*  and  4  and  Inputs  1,2,  and  3. 

They  are  described  in  the  following  subsections. 

Selected  Outputs  And  InPuts 

In  the  following  subsections  we  explain  the  four  output  and 
three  input  measures  which  were  finally  settled  on  for  inclusion 
in  our  research.  First  we  specify  the  outputs  which  are: 
completed  job  orders,  completed  work  orders,  completed  recurring 
work  actions,  and  delinquent  job  orders.  Then  we  follow  with 
descriptions  of  the  input  measures  which  are  labeled  as:  supply 
and  equipment  funding,  available  direct  labor  hours,  and 
available  passenger  carrying  vehicles. 
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Output  1:  Completed  Job  Orders  (CJO):  The  job  order  system  was 
designed  to  be  a  fast  way  to  authorize  work  that  does  not  require 
detailed  planning.  Job  orders  are  work  that  require  little  or 
no  planning,  involve  only  one  craft  shop  (e.g.,  masonry, 
electrical,  carpentry,  plumbing,  etc.),  and  materials  are 
normally  readily  available  in  bench  stock.  They  represent 
day-to-day  maintenance  and  repair  work  such  as  repair  of  air 
conditioning  units,  broken  windows,  and  minor  street  pot  holes. 
The  number  of  completed  job  orders  measures  the  amount  of 
day-to-day  work  accomplished. 

Output  2:  Completed  Work  Orders  (CWO):  Work  orders  represent 
activities  that  are  more  extensive  and  complex  than  that  done 
under  job  orders  and  usually  result  in  capitalization  of  real 
property  records.  Because  of  its  complexity,  preparation  for 
work  accomplished  under  a  work  order  involves  gathering  data  for 
review  and  analysis,  detailed  planning,  coordination  between  many 
craft  shops,  and  ordering  large  amounts  of  material.  A  work 
order  is  processed  through  production  control,  planning  and 
engineering,  material  control,  base  supply,  and  procurement.  A 
job  order  is  normally  only  processed  through  production  control. 
Examples  of  work  orders  would  be  the  construction  of  a  new  room 
or  the  complete  replacement  of  an  electrical  system. 

Output  3:  Completed  Becurring  Work  Action  (CKWA):  Hecurring 
work  items  include  recurring  (preventive)  maintenance, 
operations,  and  services  for  which  the  scope  and  level  of  effort 
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is  known  without  an  earlier  visit  to  the  job  site  each  time  the 
work  is  scheduled.  The  work  is  periodic  in  nature.  It  includes 
all  recurring  work  needed  to  prevent  breakdown  of  critical 
facilities,  equipment,  or  utilities.  Grass  cutting  and  pavement 
cleaning  (operations),  refuse  collection  and  entomology 
(services),  and  changing  air  conditioner  filters  and  preventive 
generator  maintenance  (maintenance)  are  examples  of  recurring 
work. 

Output  M:  Delinquent  Job  Orders  (DJO):  The  definition  of  a 
delinquent  job  order  depends  on  the  type  of  job  order  of  which 
there  are  three:  emergency,  urgent,  and  routine.  Thus,  in 
defining  DJO  we  first  need  to  explain  the  three  types  of  job 
orders. 

(1)  Emergency:  An  emergency  job  order  is  work  which,  if 

not  accomplished,  will  be  detrimental  to  mission  accomplishment 
or  reduce  operational  effectiveness.  This  type  of  job  order  must 
be  accomplished  within  H8  hours  of  the  identification  of  the 
requirement. 

(2)  Urgent:  An  urgent  job  order  refers  to  work  that  impacts 
mission  accomplishment  or  reduces  operational  effectiveness  less 
severely  than  work  categorized  as  emergency.  An  urgent  job  order 
is  to  completed  within  five  work  days  of  the  identification  of 
the  requirement. 
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(3)  Routine:  Routine  job  orders  include  work  that  should  be 
done  within  30  days  of  identification  of  the  requirement  if  no 
material  is  required  or  30  days  after  receipt  of  material  if 
material  is  required.  In  order  to  reduce  time  lost  to  travel, 
routine  job  orders  are  accumulated  by  geographic  area  and 
scheduled  as  work  packages  rather  than  individual  job  orders. 

We  also  need  a  category  for  scheduled  (authorized)  work 
which  was  not  completed  in  a  timely  manner.  This  work  falls  into 
a  class  called  "delinquent  job  orders". 

(*l)  Delinquent:  A  delinquent  job  order  is  one  in  any  of  the 
above  categories  that  is  not  completed  within  the  specified  time 
by  the  end  of  the  reporting  month.  Headquarters  personnel 
monitor  this  measure  to  check  the  timeliness  of  work 
accomplishment  which  they  feel  is  essential  to  maintaining 
customer  (organization  or  individual)  satisfaction.  However, 
recognizing  that  delinquent  job  orders  are  not  desired,  we  use 
its  reciprocal  as  the  measure  of  this  output.  It  is  almost  never 
the  case  that  there  are  no  delinquencies  so  we  do  not  anticipate 
a  problem  of  dealing  with  a  zero  denominator. 

Input  1:  Supply  and  Equipment  Funding  (POL):  This  is  a  supply 
support  factor.  The  larger  the  supply  and  equipment  funding,  the 
greater  the  availability  of  supplies  and  equipment  with  which  to 
accomplish  work.  This  includes  not  only  equipment  purchases  but 
also  equipment  rentals.  The  availablility  of  supplies  and 
equipment  should  reasonably  be  expected  to  affect  output 
production. 
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Input  2:  Available  Direct  Labor  Hours  (LAB  HR):  Available 
direct  labor  hours  measure  the  size  of  the  available  work  force 
which  generally  varies  proportionately  with  the  level  of  real 
property  maintenance  activity  at  each  base.  This  measure 
represents  the  amount  of  time  the  work  force  is  available  for 
accomplishing  civil  engineering  work.  LAB  HR  equals  the  total 
work  hours  available  (number  of  employees  times  the  work  week 
length)  less  an  appropriate  number  of  hours  for  sick  leave, 
vacation  time,  training,  etc. 

Input  S’-  Available  Passenger  Carrying  Vehicles  (VEH);  Another 
input  variable  is  the  number  of  passenger  carrying  vehicles 
available  to  the  base  civil  engineering  function.  Small  vans, 
pick-up  trucks,  and  station  wagons  are  examples  of  passenger 
carrying  vehicles.  Vehicles  such  as  road  graders,  back  hoes,  and 
other  specialized  equipment  are  not  included  in  this  category. 

The  measure  of  this  input  amount  was  computed  by  taking  the 
number  of  passenger  carrying  vehicles  assigned  to  the  base  real 
property  maintenance  activity  and  reducing  it  by  the  average 
vehicle  maintenance  down  time  (VDP  and  VDM).  Headquarters 
officials  believe  that  the  nonavailability  of  passenger  carrying 
vehicles  is  a  major  factor  hindering  work  completion.  Without 
vehicles,  personnel  are  not  able  to  get  to  the  work  site. 


CHAPTER  2 


ANALYSIS  AND  INTERPRETATION 

SECTION  I  -  INTRODUCTION 

In  this  section  we  turn  to  the  analysis  of  the  Air  Training 
Command's  in-house  real  property  maintenance  activities 
operational  efficiency.  We  propose  to  undertake  these 
evaluations  in  a  variety  of  ways  and  will  proceed  to  do  this  in 
the  following  manner. 

First,  we  describe  our  data  collection  procedures  and  some 
of  the  problems  encountered  such  as  the  nonavailability  of  a 
sufficient  number  of  bases  for  the  number  of  inputs  and  outputs 
to  be  used  in  the  analyses.  We  also  describe  how  we  overcome 
these  problems  through  expanding  our  data  base  via  (1)  "window 
type"  analyses  and  (2)  combining  the  data  from  two  separate  major 
air  commands  into  a  single  joint  efficiency  evaluation. 

Next  we  report  the  results  of  our  evaluation  of  the 
efficiency  of  Air  Training  Command's  operations. Our  evaluation  of 
ATC  begins  with  an  analysis  of  its  annual  data.  As  might  be 
expected  from  the  few  degrees  of  freedom  available,  most  bases 
were  rated  100%  efficient.  Therefore,  in  order  to  validate  these 
annual  efficiency  ratings  and  to  test  for  stability,  trends,  and 
other  behavior  over  time,  we  proceeded  to  window  type  analyses. 
Generally,  these  window  analyses  supported  the  annual  ratings. 

At  the  same  time,  however,  they  provided  additional  information 


on  trends  and  seasonal  behavior  which  are  not  available  from  the 
annual  data. 

Next,  we  proceed  to  combine  the  data  points  for  ATC  with 
data  from  a  Tactical  Air  Command  (TAC)  for  a  series  of  joint 
analyses  which  again  consists  of  an  analysis  of  annual  data  and 
window  type  analyses.  This  is  done  to  further  review  the  results 
obtained  from  analyzing  each  command  separately.  In  general,  the 
joint  analysis  of  annual  data  supports  our  earlier  findings  for 
ATC  while  the  joint  window  analyses  provided  new  information 
which  was  not  evident  in  the  preceding  evaluations. 

During  the  data  collection  stage  of  our  research  we  found  it 
necessary  to  reduce  the  number  of  inputs  and  outputs  in  order  to 
conform  with  the  number  of  bases  available  for  the  study. 

Therefore,  we  were  interested  in  the  effect  on  the  efficiency 
ratings  of  augmenting  our  study  with  the  deleted  inputs  and 
outputs.  This,  along  with  an  examination  of  the  sensitivity  of 
the  ratings  to  how  the  windows  for  the  window  analyses  are 
formed,  are  other  subjects  covered  in  this  chapter. 

SECTION  II  -  DATA  COLLECTION 

Our  initial  step  was  to  gather  fiscal  year  1983  annual  data, 

1  October  1982  through  September  1983,  for  each  base  within  the  j 

I 

the  two  MAJCOMs.  The  ATC  data  were  obtained  from  reports 
available  at  ATC  headquarters,  as  supplemented  and  modified  by 

i 

direct  communication  with  the  bases  which  had  filed  the  reports 
or,  in  some  cases,  following  up  to  secure  reports  which  they  had 


failed  to  file 
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Unfortunately  these  efforts  produced  complete  input  and 
output  information  on  only  seven  bases  for  ATC.  A  rule  of  thumb 
for  maintaining  an  adequate  number  of  degrees  of  freedom  when 
using  DEA  is  to  obtain  at  least  two  DMUs  for  each  input  or  output 
measure.  Note,  for  instance,  that  an  insufficient  number  of  DMUs 
for  the  variables  being  used,  would  tend  to  produce  a  result  in 
which  all  of  the  DMUs  would  be  rated  as  100%  efficient  simply 
because  of  an  inadequate  number  of  degrees  of  freedom. 

For  this  research  we  would  need  a  minimum  of  14  bases  (two 
DMUs  for  each  of  7  input  and  output  measures)  for  each  MAJCOM  to 
avoid  possibly  meaningless  results.  We  used  a  variety  of 
techniques  to  overcome  the  problem  of  having  an  insufficient 
number  of  DMUs.  The  number  of  DMUs  was  increased  through  "window 
analysis"  techniques  and/or  combining  the  data  points  from  ATC 
with  those  of  a  second  MAJCOM  into  one  overall  analysis. 

A  "window  analysis"  is  a  way  to  increase  the  number  of  DMUs, 

and  thereby,  introduce  more  degrees  of  freedom  into  an 
5a 

analysis.  The  procedures  for  a  window  analysis  involve 
subdividing  each  DMU’s  data  and  identifying  each  new  unit  as  a 
differently  dated  DMU  in  order  to  create  a  new  analysis  set  or 
"window"  from  these  subunits.  For  example,  annual  data  might  be 
broken  down  into  monthly  or  quarterly  data,  then  each  DMU  (=  Air 
Force  base)  could  be  represented  as  12  or  four  different  DMUs.  A 
moving  "window"  is  then  constructed  in  a  way  that  provides 
overlaps  and  checks  on  DMU  behavior  over  a  period  of  time.  Such 
a  moving  window  could  be  three  successive  months,  for  example,  in 
which  case  the  first  window  would  consist  of  data  on  each  DMU  for 


the  first,  second,  and  third  months.  The  second  window  would 
consist  of  data  on  each  DMU  for  the  second,  third,  and  fourth 
months,  and  so  on. 

Note  that  the  data  for  month  two  is  used  twice,  once  in  the 
first  window  (months  one,  two,  and  three)  and  again  in  the  second 
window  (months  two,  three,  and  four).  This  provides  a  two  way 
comparison  for  each  DMU  relative  to  its  efficiency  ratings  (and 
sources  of  inefficiencies)  from  two  different  sets  of  data. 

Thus,  moving  over  time  one  can  check  for  stability,  trends, 
seasonal  behavior,  or  other  properties  of  potential  interest. 
Moreover,  further  insight  can  be  supplied  by  additional 
comparisons  with  annual  data  or  other  ways  of  forming  the 
windows,  and  so  on. 

Observed  Inputs  and  Outputs 

Tables  A. 1  and  A. 2  in  Appendix  A  show  the  observed  input  and 
output  values  used  in  this  study  for  ATC  and  MAJCOM  //2.  These 
tables  contain  annual  and  quarterly  input  and  output  values  for 
Fiscal  Year  (=  FY)  83* 

in  order  to  extend  the  window  analysis,  we  also  obtained 
data  on  the  first  quarter  of  FY  84  which  are  also  shown  in  Tables 
A. 1  and  A. 2.  In  a  related  attempt  to  extend  the  significance  of 
the  window  analyses  we  also  attempted  to  get  data  for  FY  83  by 
month.  But  this  information  was  unavailable  for  some  of  the 
bases  and  is  therefore  not  included  in  our  work. 


SECTION  III  -  ANALYSIS  OF  ATC 


Analysis  Using  Annual  Data 

We  initiate  our  efficiency  evaluation  of  ATC  by  using  annual 
data  for  the  seven  input  and  output  measures  previously  discussed 
in  a  DEA.  As  expected,  there  was  little  discrimination  between 
the  bases  on  their  relative  efficiencies  since  there  were  so  few 
DNUs  relative  to  the  number  of  inputs  and  outputs  used.  Using 
only  annual  data,  all  bases  were  either  rated  as  100%  efficient 
or  very  close  to  100%  efficient.  That  is,  they  had  values  of 

A  Rh 

I$q  s  0*  s  1  and  slack  variable  values  were  zero. 

Table  1  shows  these  results  for  ATC.  Under  the  efficiency 
rating,  hQ  in  column  1,  the  slack  variables  for  the  three  inputs 
and  four  outputs  used  in  our  analyses  are  listed,  while  columns  2 
through  8  display  the  optimal  values  obtained  for  each  of  those 
variables.  The  slack  variables  which  appear  under  the  efficiency 
rating  in  each  column  relate  to  the  input  or  output  measure  with 
which  the  slack  variable  is  associated.  For  example,  the  3  in 
column  3  opposite  VEH  is  the  value  of  the  slack  variable  (5Vfc;H) 
in  the  constraint  associated  with  the  passenger  carrying  vehicle 

input  measure  for  Lowry  AFB.  The  value  of  $1,166,502  in  column  M 

_  * 

represents  the  amount  of  slack  (5DqL)  in  the  constraint  for  the 
supply  and  equipment  funding  input  measure  for  Mather  AFB. 

Note  from  Table  1  that  Keesler,  Reese,  Sheppard,  Vance,  and 


Williams  are  rated  100%  efficient.  Their  optimal  solutions  have 
0*  *  1  and  all  slack  values  are  zero. 
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Table  1 


Effeciency  Measure  Values  -  ATC 


(1) 

Efficiency 
Measure  Name 

(2)  (3) 

Kees ler  Lowry 

(4)  (5)  (6) 

Efficiency  Measure  Values 
Mather  Reese  Sheppard 

(7) 

Vance 

Williams  § 

h*  *  0* 

1.0 

.915 

.975 

1.0 

1.0 

1.0 

1.0  1 

Slack 

Variables 

(si  ,  sr  ) 

VEH 

★ 

3 

★ 

★ 

* 

★ 

★ 

DOL 

* 

* 

1166502 

★ 

* 

* 

★ 

LAB  HR 

■k 

72461 

* 

★ 

k 

*• 

k 

CWO 

* 

6 

105 

★ 

k 

* 

k 

CJO 

* 

129 

* 

* 

k 

* 

k 

CRWA 

* 

* 

* 

★ 

k 

* 

k 

DJO 

* 

** 

* 

★ 

k 

* 

1 

*  s 

Note 

1.  *  Indicates 

2.  **  Indicates 

there  was 
there  was 

no  positive  slack, 
positive  slack  values 

associated  with 

delinquent  job  orders.  We  do  not  snow  the  numerical  value 
because  It  has  no  intuitive  meaning  since  we  are  using  a 
recipricol . 

Legend 

Inputs: 

VEH  -  Passenger  Carrying  Vehicles 
DOL  -  Supply  and  Equipment  Funding 
LAB  HR  -  Available  Direct  Labor  Hours 
Outputs: 

CWO  -  Completed  Work  Orders 
CJO  -  Completed  Job  Orders 
CRWA  -  Completed  Recurring  Work  Actions 

DJO  -  Delinquent  Job  Orders 


Also  observe  from  this  table  that  operational  inefficiencies 
within  an  organization  are  identified  in  two  parts  via  L)EA.  One 
part  is  the  6*  values  and  the  other  part  is  the  optimal  slack 
variable  values  (s^  ,  s*  ).  The  6*  indicates  scale  and 
technical  inefficiencies  while  slack  values  represent  mix 
inefficiencies  (either  input  or  output). 

Lowry  and  Mather  are  rated  less  than  100%  efficient  by  DEA 
on  two  grounds:  first,  they  have  0*  <  1  and  second  they  also 
have  non-zero  slack  variable  values.  I.e.,  Lowry  has  0*  =  .915 

ft  ft  ft  ft 

and  slack  variables  =  3,  s^AB  hr  =  72,^61,  s£WQ  =  6  and  s£J0 

=  129-  This  means  that  Lowry  was,  at  best,  only  91.5%  efficient 
relative  to  the  reference  set  data  and  thus  should  be  able  to 
reduce  all  of  its  inputs  by  8.5%  and  still  produce  the  same  level 
of  output.  In  addition,  particular  inputsw  (VEH  and  LAB  HH)  can 
be  further  reduced  by  the  input  slack  variable  values  without 
effecting  a  reduction  in  output.  Finally,  even  with  all  input 
reductions  made,  the  outputs  can  still  be  increased  by  their 
slack  values.  Only  after  all  of  these  adjustments  have  been  made 
will  the  base  be  efficient  and  at  its  Most  Productive  Scale  Size 
(average  productivity  is  maximized). 

Continuing  on,  we  note  the  large  slack  variable  value  for 
supply  and  equipment  funding  (DOL)  for  Mather.  It  is  over  one- 
third  of  the  actual  amount  of  supply  expenses  incurred  by  that 
base  which  implies  that  the  .975  efficiency  rating  is  too  high. 

As  previously  noted,  Vance  was  rated  100%  efficient. 

However,  there  is  evidence  that  Vance's  efficient  rating  may  be 
due  to  special  features  in  its  operations.  DEA  provides  a  basis 
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for  relative  efficiency  evaluations  in  that  efficient  DMUs  should 
generally  appear  in  the  reference  set  for  other  DMUs.  DMUs  rated 
efficient  which  do  not  appear  in  the  efficiency  set  of  other  DMUs 
may  not  actually  be  comparable  with  any  of  the  other  DMUs.  This 
is  the  case  for  Vance.  It  does  not  appear  as  a  member  of  any 
inefficient  unit's  optimal  basis  set,  and  therefore,  warrants 
further  investigation  before  it  should  be  considered  100% 
efficient.  Williams,  on  the  other  hand,  was  in  the  optimal  basis 
set  for  both  inefficient  units.  This  lends  support  to  Williams 
efficient  rating  since  it  "dominates"  each  inefficient  unit  in 
one  or  more  dimensions. 

Window  Analyses  For  ATC 

The  above  analysis  is  only  a  start,  by  breaking  annual 
Information  into  quarterly  data  and  undertaking  window  types  of 
analyses,  we  can  obtain  a  series  of  efficiency  ratings  for  each 
base's  quarterly  operations.  Recall  from  our  previous  discussion 
in  this  report  that  window  analyses  allow  us  to  check  the 
validity  of  the  annual  ratings  while  obtaining  new  information  on 
trends,  seasonal  behavior,  and  stability  within  the  data. 

With  five  quarters  of  data,  we  are  able  to  perform  three 
separate  three-quarter  window  analyses  which  we  refer  to  us 
Analyses  #1,#2,  and  #3.  Analysis  #1  consists  of  data  from  each 
base  for  the  first,  second,  and  third  quarters  while  Analysis  #2 
has  second,  third,  and  fourth  quarter  data,  and  Analysis  used 
data  from  the  third,  fourth,  and  fifth  quarters. 


nano 


26 


We  display  partial  results  from  these  analyses  as  in  Table 

y  » 

2.  Here  in  columns  3  through  5  we  have  the  hQ  (efficiency 
measure)  value  for  each  quarter  in  a  particular  analysis.  For 
easy  comparison,  we  show  the  hQ  resulting  from  the  analysis  of 
annual  data  in  column  2.  For  example,  Lowry's  quarterly 
operations  in  Analysis  #1  received  ratings  of  87.4%,  78.9%,  and 
100%  efficient  for  the  first,  second,  and  third  quarters 
respectively.  Also  note  that  Lowry's  second  quarter  is  a  DhU  in 
Analysis  #2.  From  this  new  reference  set,  Lowry's  second  quarter 
earned  an  efficiency  rating  of  69.0%.  All  of  these  quarterly 
ratings  can  be  compared  to  Lowry's  annual  efficiency  rating  of 
91.5% 

We  begin  our  review  of  the  results  of  the  window  analyses 
reported  in  Table  2  with  Keesler.  The  results  of  the 
three-quarter  window  analyses  generally  support  Keesler 's  rating 
as  efficient  on  the  basis  of  its  annual  data.  Table  2  shows 
Keesler  with  several  ratings  of  less  than  one  but  all  of  these 
are  at  efficiency  values  of  .95  or  higher.  One  might  reasonably 
expect  some  variation  in  ratings  due  to  changes  in  time  periods 
and  reference  sets,  and  the  range  of  variations  (5%  or  less)  for 
Keesler  all  seem  reasonable. 

Lowry  and  Mather  were  generally  inefficient  across  all 
window  analyses.  This  conforms  to  the  inefficiency  ratings 
obtained  when  using  solely  annual  data.  Furthermore,  the  window 
analyses  supplies  additional  details  which  indicate  a  downward 
trend  in  efficiency  for  Mather  with  consistently  low  values  in 
the  third  quarter. 
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Table  2 

ATC  Efficiency  Ratings 
Window  Analysis  Using  Three  Quarters 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

Base 

Annual 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

Keesler 

1.0 

Analysis 

#1 

1.0 

.963 

1.0 

Analysis 

if  2 

.945 

.994 

1.0 

Analysis 

if  3 

.957 

•  961 

1.0 

Lowry 

.915 

Analysis 

if 1 

.874 

.789 

1.0 

Analysis 

if  2 

.690 

.968 

.791 

Analysis 

if  3 

.828 

.768 

1.0 

Mather 

.975 

Analysis 

if 1 

1.0 

.849 

.640 

.w 

Analysis 

if2 

.928 

.645 

Analysis 

if  3 

.645 

.836 

.838 

Reese 

1.0 

Analysis 

#1 

1.0 

.887 

.875 

Analysis 

if  2 

1.0 

.967 

.827 

Analysis 

if  3 

1.0 

.850 

1.0 

Sheppard 

1.0 

.916 

Analysis 

#1 

.919 

1.0 

M 

Analysis 

if  2 

.893 

1.0 

Analysis 

»3 

1.0 

1.0 

1.0 

Vance 

1.0 

Analysis 

#1 

1.0 

1.0 

1.0 

Analysis 

if  2 

1.0 

1.0 

1.0 

Analysis 

#3 

1.0 

1.0 

1.0 

Williams 

1.0 

Analysis 

#1 

1.0 

'  .952 

1.0 

Analysis 

if 2 

.899 

1.0 

1.0 

Analysis 

«3 

1.0 

1.0 

1.0 

RATINGS 

1775  =  Efficiency 

<1.0  =  Inefficiency 
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Lowry's  efficiency  rating  was  significantly  higher  in  the 
third  quarter  than  in  the  other  quarters.  Turning  to  Table  A. 1 
in  Appendix  A  we  see  that  Lowry's  output  during  that  period  was 
significantly  better  that  in  the  other  periods.  This  may 
indicate  a  reporting  error  such  as  double  counting  of  some  work, 
i.e.,  the  same  job  order  was  reported  as  being  completed  twice. 

We  checked  with  base  civil  engineering  personnel  to  see  if  such 

i 

possibilities  did  occur.  They  responded  that  no  reporting 
inconsistencies  existed.  Hence,  other  causes  might  need  to  be 
investigated  such  as  reporting  work  being  completed  in  this 
quarter  which  had  been  substantially  completed  in  prior  periods. 

Table  2  provides  evidence  that  Reese's  efficiency  rating  is 
unstable  and  depends  on  the  time  period  and  reference  set.  It 
has  several  quarterly  ratings  which  are  significantly  (more  than 
10%)  below  its  annual  rating  of  hQ  s  1.  Nevertheless,  there  are 

also  several  ratings  which  equal  the  annual  efficient  rating  of 

« 

hQ  =  1 .  bach  of  these  efficient  ratings  was  checked  for  the 

presence  of  positive  slack  values  which  would  imply  a  mix 

n 

inefficiency  that  is  not  evident  in  the  h^  value.  However,  all 

* 

quarterly  operations  with  hQ  =  1  also  had  all  slack  variable 
values  equal  to  zero.  In  addition,  these  quarterly  operations 
appeared  in  the  efficient  reference  set  of  inefficient 
operations.  Thus,  the  efficient  ratings  received  by  Reese's 
quarterly  operations  appears  valid. 

The  instability  of  the  ratings  for  Reese  is  not  only  evident 
from  the  variation  in  ratings  between  quarters  but  also  from  the 
large  variation  in  the  ratings  within  each  quarter.  For 
example,  see  the  third  quarter. 
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*  Sheppard's  efficient  rating  calculated  from  annual  data 
appears  to  be  influenced  by  its  strong  performance  during  the 
last  two  quarters  of  the  fiscal  year.  As  can  be  seen  from  Table 
2,  there  was  a  marked  improvement  in  Sheppard’s  performance 
between  the  second  and  third  quarters.  This  suggested  further 
inquiry  and  so  we  investigated,  along  with  MAJCOM  officials, 
whether  there  had  been  a  change  of  command  or  civil  engineer  or 
some  other  action  at  this  base  which  might  account  for  the  better 
performance.  However,  nothing  was  apparent  from  this 
headquarters-level  inquiry  and  perhaps  a  further  base-level 
inquiry  should  be  undertaken. 

Vance's  efficient  rating  from  using  annual  data  is  supported 
by  the  window  analysis.  This  base  was  rated  efficient  regardless 
of  the  reference  set  and  regularly  appeared  in  the  optimal  basis 
sets  of  inefficient  units.  It  is  interesting  to  note  that  the 
real  property  maintenance  activity  (RPMA)  at  this  base  is 
contracted  out  by  the  Air  Force,  and  the  only  Air  Force  personnel 
assigned  to  this  base's  RPMA  are  contract  monitors. 

ATC  officials  felt  that  there  are  two  reasons  for  Vance's 
efficient  rating.  First,  almost  the  entire  contractor's  work 
force  consists  of  retired  military  civil  engineering  personnel 
which  results  in  a  highly  skilled  work  force  compared  to  the 
other  bases  which  have  several  real  property  maintenance 
trainees.  This  finding  indicates  that  we  may  have  missed  an 
important  RPMA  output  —  the  training  of  personnel. 

The  second  advantage  the  contractor  has  is  that  he  receives 
his  supply  and  equipment  funding  at  the  beginning  of  the  fiscal 
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•year  while  the  other  bases  receive  their  funding  sporadically 
throughout  the  year.  This  permits  the  contractor  to  better 
program  and  use  his  resource  since  he  knows  the  amount  of 
resources  he  will  have  on  hand  for  the  year  while  other  base 
civil  engineers  operate  under  uncertainty. 

Williams  also  appears  to  be  performing  well.  Except  for  a 
slight  aberration  in  the  second  quarter,  it  was  consistently 
rated  efficient.  William's  fourth  quarter  execution  was 
especially  strong.  This  DHU  was  in  the  tom?«uri%or**  set  for  all 
inefficient  units  —  12  out  of  12  inefficient  DMUs  in  Analysis  #2 
and  seven  out  of  seven  inefficient  units  in  Analysis  #3* 

Finally,  note  from  Table  2  that  there  is  a  tendency  for  more 
efficient  performance  by  the  bases  in  the  first  quarter  of  the 
fiscal  year.  The  first  quarter  shows  five  of  seven  bases  rated 
efficient  and  the  fifth  quarter  (=  first  quarter  FY  84)  has  six 
of  seven  bases  rated  efficient.  Ho  other  quarter  has  such  a 
great  percentage  of  efficient  DMUs. 

On  the  other  hand,  the  second  quarter  has  the  worst 
performance.  Only  three  of  14  DMUs  were  rated  efficient. 

We  discussed  with  ATC  officials  possible  reasons  for  this 
cyclical  result  such  as  seasonal  variation  due  to  weather  or 
fluctuation  in  funding  levels.  It  was  their  opinion  that  this 
could  result  from  Congress'  consistent  failure  to  appropriate 
funds  at  the  beginning  of  the  fiscal  year.  During  the  first 
quarter  of  the  fiscal  year,  the  bases  are  not  aware  of  what  their 
programs  will  be  for  the  year  because  they  are  operating  under  a 
Continuing  Kesolution  Authority.  Thus  they  order  few  new 
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*  supplies  and  equipment  and  use  what  they  have  on  hand.  Sinoe  our 

input  measure  is  supply  and  equipment  expenses  the  above  actions 

would  result  in  lower  expenses  and  hence  possibly  a  higher 

7 

efficiency  rating. 

The  effect  of  this  inventory  drawdown  continues  on  into  the 
second  quarter  but  has  the  opposite  impact.  Normally  by  the 
second  quarter  bases  know  what  funds  are  currently  available  to 
them  for  the  year,  and  thus  they  begin  to  build  their  inventories 
back  up.  This  results  in  higher  than  normal  expenses  and  can 
contribute  to  a  lower  efficiency  rating  since  the  supply  and 
equipment  funding  (expense)  input  would  overstate  the  amount  of 
supplies  actually  consumed. 

This  cyclical  variation  also  reflects  the  programming 
advantage  that  Vance  possibly  has  in  receiving  its  supply  and 
equipment  finding  at  the  beginning  of  the  fiscal  year  which  was 
previously  discussed.  Note  that  of  the  three  decision  making 
units  which  were  rated  100J  efficient  for  the  second  quarter,  two 
of  them  were  Vance's  second  quarter  operations.  Thus  it  appears 
that  Vance  is  the  only  base  that  is  unaffected  by  the  lack  of  an 
appropriation  at  the  beginning  of  the  fiscal  year  and  the 
corresponding  impact  on  programming  the  use  of  its  resources  in 
an  optimal  (efficient)  manner.  Hence  this  particular  advantage 
of  Vance's  contract  operations  is  supported. 

There  is  one  other  bit  of  information  provided  by  DEA  that 
is  managerially  useful  and  can  aid  further  investigations.  As 
discussed  in  Section  il  of  Chapter  1,  DEA  identifies  the 
efficient  DMUs  (=bases  or  quarterly  operations)  with  which  the 
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•  « 

•  inefficient  base  is  compared  in  arriving  at  its  efficiency 
rating. 

To  illustrate  what  we  are  referring  to  here,  we  will  use  the 
results  of  the  analysis  using  annual  data.  Recall  that  Lowry  and 
Mather  were  rated  inefficient  in  this  analysis  while  Keesler, 
Reese,  Sheppard,  Vance,  and  Williams  were  rated  efficient.  The 
comparison  set  for  Lowry  consisted  of  the  efficient  bases  — 
Keesler,  Reese,  Sheppard,  and  Williams  —  while  Mather  was 
compared  solely  to  Williams.  Thus,  the  civil  engineers  at  the 
inefficient  bases  and  other  interested  individuals  can  look  at 
the  operations  of  the  comparison  set  bases  as  a  source  of 
information  for  correcting  their  operation's  inefficiencies.  In 
Lowry's  case  this  might  not  be  meaningful  or  easily  accomplished 
due  to  the  large  number  of  bases  involved.  However,  the  civil 
engineer  at  Mather  might  find  it  very  beneficial  since  there  is 
only  one  base  in  his  comparison  set. 

The  findings  discussed  in  this  section  are  indicators  of 
operational  efficiency  (or  inefficiency)  in  the  real  property 
maintenance  function  of  the  bases  reviewed.  These  indicators  are 
a  means  to  an  end,  which  is  efficient  operations,  and  not  the  end 
in  themselves.  As  such,  the  information  serves  as  guides  to 
management  for  additional  investigation. 

As  discussed  for  Lowry  and  Sheppard,  we  initiated  some 
preliminary  investigations  from  the  headquarters  level.  However, 
these  investigations  generally  proved  to  be  inconclusive  or 
tentative  and  hence  further  field  (base)  follow-up  is  required. 
Such  field  follow-up  could  be  accomplished  by  base  personnel, 
staff  auditors,  or  headquarters  staff  assistance  visits. 
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Summary  of  ATC  Findings 

He  began  our  evaluation  of  ATC's  base  real  property 
maintenance  by  analyzing  base  operations  via  annual  data.  As 
expected,  all  bases  were  rated  100%  efficient  or  nearly  100% 
efficient  due  to  having  an  insufficient  number  of  bases  in  the 
analysis  for  the  number  of  inputs  and  outputs  used. 

To  test  the  stability  and  validity  of  the  annual  efficiency 
ratings  and  to  search  for  any  underlying  trends,  we  performed  a 
series  of  window  analyses.  We  found  that  this  type  of  analysis 
could  be  used  not  only  to  substantiate  or  question  the  results  of 
the  analysis  of  annual  data  but  also  to  provide  additional 
insight  into  the  operation  of  the  bases.  For  example,  it 
indicated  a  downward  trend  in  efficiency  for  Mather. 

Finally,  these  results  are  indicators  of  efficiency  or 
inefficiency  and  should  be  used  as  guides  for  further 
investigation  at  the  headquarters  and  base  level  of  operations. 


SECTION  IV  -  JOINT  ANALYSIS  OF  ATC  AND  TAC 

To  further  review  and  validate  the  results  obtained  for  ATC 
from  the  previous  analyses,  we  now  bring  in  data  of  nine  bases 
from  a  second  major  air  command,  TAC,  and  combine  these  data  with 

O 

ATC's  data  for  joint  analysis.  With  this  approach  we  are  able 
to  introduce  more  observations  into  an  analysis  which 
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•should  provide  better  efficiency  evaluations.  Even  if  the  two 
sets  of  data  are  not  quite  comparable  this  should  show  up  in  a 
consistent  separation  of  their  efficiency  evaluations. 

To  further  validate  (and  justify)  our  joint  analyses,  we 
investigated  the  real  property  maintenance  activities  of  the  two 
commands  and  could  not  find  any  real  source  of  possible  troubles 
for  the  kinds  of  analyses  we  are  conducting.  The  dimensions  we 
are  looking  at  are  the  same  across  all  major  air  commands. 
Maintenance  of  facilities,  systems,  and  equipment  are  similar 
between  the  MAJCOMs.  Both  commands  are  required  to  follow  the 
same  Air  Force  regulations  and  directives  regarding  real  property 
maintenance.  Thus,  base  civil  engineers  from  each  MAJCOM  should 
interpret  the  input  and  output  measures  similarly  and  follow  the 
same  procedures  regardless  of  the  primary  mission  of  the  base/ 

Joint  Analysis  Of  Annual  Data 


Table  3  compares  the  efficiency  ratings  calculated  from 
using  annual  data  when  the  Major  Air  Commands  were  analyzed 
separately  (columns  2  and  3)  and  when  they  are  combined  (column 
H).  In  the  following  discussion  we  highlight  ATC  in  order  to 
maintain  continuity  with  our  earlier  discussions.  As  can  be  seen 
from  this  table,  several  ATC  bases  had  their  efficiency  ratings 
effected  by  this  joint  analysis. 

Those  bases  that  had  their  annual  efficiency  ratings  ! 

unchanged  or  changed  by  less  than  5 %  received  strong  support  that 
the  original  ratings  were  valid  since  the  introduction  of  ■ 
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Table  3 


Comparison  Of  Efficiency  Ratings  Prom  Annual  Data 


(1) 

Base 

(2) 

ATC 

(3) 

MAJCOM  #2 

(4) 

ATC  &  #2 

Keesler 

1  .0 

.975 

Lowry 

.915 

.915 

Mather 

.975 

.728 

Reese 

1  .0 

1  .0 

Sheppard 

1  .0 

.883 

Vance 

1  .0 

1  .0 

Williams 

1  .0 

1  .0 

Luke 

1  .0 

.949 

Howard 

.927 

.927 

Langley 

.660 

.645 

George 

1.0 

1  .0 

Moody 

1  .0 

.967 

Shaw 

1  .0 

.518 

Myrtle  Beach 

1 .0 

1  .0 

Holloman 

.624 

.551 

Bergstrom 

1  .0 

.627 

additional  observations  did  little  to  change  those  rating.  This 
would  include  Kessler,  Lowry,  Reese,  Vance,  and  Williams. 


However,  Vance  still  seems  to  be  an  anomaly.  Although  it 

continues  to  be  rated  efficient,  it  again  did  not  appear  in  any 

of  the  optimal  basis  sets  for  inefficient  units  which,  as 

previously  discussed,  is  an  indication  that  the  base  is  a  self- 

evaluator  and  requires  further  review  before  it  should  be 

considered  efficient.  Although  by  this  time  in  our  study, 

possibly  for  special  reasons  which  make  it  not  quite  comparable 

with  other  bases  there  are  strong  indications  that  Vance  is 

operating  efficiently,  at  least  in  its  own  special  way. 

Mather  and  Sheppard  had  their  efficiency  ratings  (based  on 

annual  data)  reduced  by  more  than  10%.  Note  especially  the 

results  for  Mather.  The  annual  slack  values  and  previous  window 

analyses  had  already  suggested  that  the  .975  efficiency  rating 

g 

for  Mather  based  on  annual  data  was  probably  too  high.  The  .728 
rating  received  in  the  joint  analysis  appears  to  be  more  in 
conformance  with  the  results  of  the  preceding  window  analyses. 
Sheppard's  results  indicated  inefficiencies  that  were  not 
identified  in  the  previous  analyses  and  thereby  provide  new 
information  for  study  and  analysis. 

Window  Analyses  Combining  ATC  and  TAC 

To  further  test  the  results  obtained  from  this  expanded 
analysis  of  annual  data,  we  turn  to  the  window  analysis  technique 
used  in  the  previous  sections.  Take  4  presents  partial  results 
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Table  4 

ATC  Efficiency  Ratings  -Combined  Data 


Window 

Analysis 

Using  Three  Quarters 

(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

Base 

Annual 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

Keesler 

.975 

Analysis 

04 

.847 

.  821 

.863 

1 

Analysis 

it  5 

.769 

.  825 

.829 

Analysis 

a  6 

.761 

.769 

.955 

Lowry 

•  915 

t 

Analysis 

#4 

.670 

.656 

.999 

Analysis 

it  5 

.636 

.955 

.753 

Analysis 

it  6 

.696 

.614 

1.0 

Mather 

•  728 

1 

1 

Analysis 

#4 

.892 

.650 

.466 

Analysis 

it  5 

.672 

.459 

.567 

Analysis 

it  6 

.445 

.571 

.776 

l 

Reese 

1.0 

1 

1 

Analysis 

#4 

1.0 

.838 

.740 

Analysis 

itb 

1.0 

.897 

.757 

Analysis 

it  6 

.579 

.547 

.935 

i 

Sheppard 

.883 

1 

j 

j 

Analysis 

#4 

.741 

.742 

.918 

1 

Analysis 

it  5 

.651 

.819 

.825 

< 

Analysis 

itb 

.832 

.797 

.777 

Vance 

1.0 

Analysis 

#4 

.855 

.883 

.824 

f 

Analysis 

//5 

1.0 

.966 

1.0 

f 

Analysis 

it  6 

.893 

.873 

1.0 

j 

Williams 

1.0 

Analysis 

it  4 

.960 

.790 

1.0 

i 

Analysis 

it  5 

.794 

1.0 

1.0 

Analysis 

il  6 

.847 

.826 

.972 
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*  „  ‘(h^  values)  from  this  series  of  analyses  in  the  same  format  as 

Table  2. This  series  of  window  analyses  are  referred  to  as 
Analysis  ilk  (first,  second,  and  third  quarters),  Analysis  11 5 
(second,  third,  and  fourth  quarters)  and  Analysis  #6  (third, 
fourth,  and  fifth  quarters). 

Reviewing  Table  4  we  see  that  the  results  reported  in  Table 
3  are  substantiated  for  some  of  the  bases.  The  ratings  for 
Mather  and  Williams  show  some  fluctuation  but  generally  support 
the  annual  rating  as  a  reflection  of  the  organization 's  overall 
performance  for  the  year.  A  few  additional  findings  concerning 
these  units  also  are  noteworthy. 

Through  the  third  quarter,  Mather  again  shows  a  definite 
downward  trend  in  the  efficiency  of  its  operations  with  the  third 
quarter  being  extremely  poor.  The  fourth  quarter  and  fifth 
quarter  (=first  quarter  FY  84)  show  some  improvement  but  the  base 
does  not  recover  to  its  performance  in  the  first  quarter  of 
FY  83.  This  same  trend  was  reflected  in  Table  2,  but  it  was  not 
as  pronounced. 

The  second  quarter  efficiency  diminution  for  Williams 
evident  in  Table  2  is  also  present  here  but  it  is  even  more 
pronounced . 

The  results  for  Vance  are  inconclusive  and  require  further 

* 

investigation.  Reviewing  the  hp  values  listed  in  Table  4  we  see 

that  these  values  for  Vance  are  generally  .87  and  higher  and  the 

1 1 

mean  for  all  of  these  values  is  .91.  From  this  information  we 
might  conclude  that  the  relatively  efficient  rating  resulting 
from  annual  data  is  valid. 
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However,  as  was  true  with  previous  analyses,  we  again  have 
indications  that  Vance's  operations  might  be  less  efficient  than 
suggested  by  the  ratings.  Out  of  eight  possible  FY  83  quarterly 

ratings  in  the  window  analyses,  Vance  had  only  two  which  were 

» 

hQ  =  1.  both  of  these  ratings,  second  and  fourth  quarter  in 
Analysis  #2,  are  suspect  since  they  are  in  only  one  optimal  basis 
set  which  happens  to  be  Vance's  own  third  quarter.  Thus  Vance 
must  continue  to  be  regarded  as  a  self-evaluator.  If  we 
disregard  (eliminate)  the  second  and  fourth  quarter  ratings  in 
Analysis  02,  we  would  have  a  mean  rating  of. 88  for  Vance.  This 
is  12%  below  the  annual  rating  of  100%  efficient  and  is  probably 
a  greater  variation  than  reasonably  could  be  expected  from 
changes  in  time  periods  and  reference  sets. 

For  the  remaining  bases,  Keesler,  Lowry,  Reese,  and 
Sheppard,  there  is  evidence  that  the  annual  efficiency  rating  may 
be  overstated  and  the  window  analyses  have  uncovered  additional 
inefficiencies.  The  ratings  computed  via  the  window  analyses  are 
consistently  lower  than  the  annual  measures  by  significant 
amounts.  For  example,  Lowry's  average  quarterly  efficiency 
ratings  from  Table  4  of  .67  for  the  first  quarter,  .646  for  the 
second  quarter,  .883  for  third  quarter,  and  .684  for  the  fourth 
quarter  are  well  below  the  rating  of  .915  for  the  year. 

Other  findings  discussed  in  analyzing  ATC  separately  are 
also  evident  here.  Lowry's  abnormally  high  third  quarter 
productivity  is  again  evident.  Sheppard's  trend  to  improved 
performance  beginning  with  the  third  quarter  is  also  reflected. 
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The  joint  window  analyses  also  provides  new  information  on 
Reese's  real  property  maintenance  function.  It  highlights  a 
trend  that  was  not  apparent  from  the  previous  analyses.  There  is 
a  definite  downward  trend  in  Reese's  operational  efficiency  as 
can  be  seen  from  the  following  average  quarterly  efficiency 
ratings:  1st  -  1.0,  2nd  -  . 9 1 9 1  3rd  -  .79,  and  4th  -  .652. 


Contribution  Of  The  Joint  Analysis 


The  purpose  of  combining  the  data  from  two  separate  major 
air  commands  into  a  single  analysis  was  to  introduce  more 
observations  into  our  research  and  thereby  obtain  additional 
insight  into  our  DEA  evaluations.  Additional  light  is  also  shed 
on  the  evaluations  obtained  from  the  annual  data  or  window 
analyses  for  each  separate  command.  We  can  confirm  some  of  the 
previous  findings  and  raise  doubts  about  others  in  ways  that 
provide  additional  ways  of  identifying  possible  inefficiencies 
and  trends  that  were  concealed  or  not  uncovered  in  the  preceding 
analyses . 

In  our  research,  the  joint  analyses  conformed  to  the 
previous  findings  for  ATC  in  several  cases.  In  both  the  joint 
analyses  and  the  individual  analyses,  Lowry  received  the  same 
efficiency  rating  based  on  annual  data.  Also,  the  window  type 
analyses  from  both  approaches  highlighted  Lowry's  greater  than 
normal  third  quarter  efficiency. 

For  Mather,  the  combined  analysis  of  annual  data  supported 
the  implication  from  ATC's  window  analyses  that  Mathers  annual 
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analyses  also  identified  the  same  downward  trend  in  efficiency 
for  Mather  that  was  identified  in  the  ATC  window  analyses. 

The  combined-data  analyses  (both  annual  and  window  analyses) 
also  uncovered  inefficiencies  that  were  apparent  when  using 
solely  ATC  data.  Keesler  and  Sheppard  were  assigned  diminished 
efficiency  ratings.  In  addition,  these  combined-data  analyses 
brought  forth  the  downward  trend  in  efficiency  for  Reese. 

Although  we  combined  the  data  from  two  different  major  air 
commands  for  our  research,  this  type  of  analysis  is  not  readily 
implementable  in  the  Air  Force  without  a  change  in  current  real 
property  maintenance  management  procedures.  The  Air  Force 
delegates  operational  oversight  of  the  real  property  maintenance 
function  to  the  major  air  command  while  the  base  has  operational 
control.  Hq  Air  Force  directorates  are  not  involved  with  the 
operations  of  base-level,  in-house  real  property  maintenance 
activities  and  as  a  result,  there  are  no  procedures  for 
aggregating  real  property  maintenance  work-load  data  at  Air  Force 
headquarters.  At  the  same  time,  currently  there  are  no 
procedures  for  exchanging  this  type  of  information  between  major 
air  commands.  Thus,  for  this  kind  of  joint  evaluations  to  be 
possible,  major  air  command  officials  must  be  willing  to  share 
work-load  and  resource  consumption  information  between  commands 
and  establish  procedures  which  will  permit  this  exchange. 


SECTION  V  -  SENSITIVITY  OF  ATC  EVALUATIONS 


In  this  section  we  test  the  sensitivity  of  our  program 
evaluations  for  ATC  bases  to  changes  in  the  categories  of  inputs 
and  outputs  used  in  the  analyses.  Recall  that  during  our  data 
collection  phase  we  found  it  necessary  to  reduce  the  number  of 
inputs  and  outputs  included  in  our  analysis  due  to  having  an 
insufficient  number  of  DMUs.  Here  we  will  check  the  effect  on 
the  preceding  efficiency  ratings  of  augmenting  our  study  with 
some  of  the  inputs  and  outputs  that  were  previously  deleted. 

We  will  also  review  the  sensitivity  of  our  evaluations  to 
how  we  form  the  windows  for  the  window  analyses. 

Changes  In  Inputs  And  Outputs 

In  accomplishing  this  research  we  found  it  necessary  to 
eliminate  some  inputs  and  outputs  from  our  analyses.  The  primary 
reason  for  making  these  reductions  was  due  to  an  insufficient 
number  of  DMUs. 

Recourse  to  a  window  analysis  increased  the  number  of  DMUs 
and  allowed  us  to  check  the  stability  of  the  efficiency  ratings. 

This  same  technique  also  enables  us  to  test  the  sensitivity  of 

* 

the  efficiency  rating,  hQ,  to  changes  in  the  inputs  and  outputs 
used  to  effect  these  evaluations. 

Since  we  have  21  DMUs  with  the  three-quarter  window 
analysis,  we  are  able  to  provide  an  adequate  number  of  degrees  of 
freedom  to  accommodate  a  Data  Envelopment  Analysis  with  ten 
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’inputs  and  outputs.  Therefore  we  generated  two  more  analyses 

which  we  refer  as  Augmented  Study  #1  (AS1)  and  Augmented  Study  02 

( AS2) .  AS1  used  the  same  inputs  and  outputs  as  Analysis  #2  but 
12 

augmented  them  with  three  additional  outputs:  percentage  of 
job  orders  completed  (%CJO),  percentage  of  work  orders  completed 
(%CWO),  and  percent  of  recurring  work  actions  completed  (%CRWA). 
AS2  had  the  same  inputs  and  outputs  as  Analysis  #2  plus  three 
additional  inputs:  number  of  available  job  orders  (AJO),  number 
of  available  work  orders  (AWO),  and  the  number  of  scheduled 
recurring  work  actions  (SRWA). 

The  observed  values  for  the  additional  input  and  output 
measures  used  in  these  augmented  studies  are  listed  in  Table  5* 
Values  are  presented  for  the  second,  third,  and  fourth  quarters 
so  to  be  comparable  to  Analysis  #2.  Using  Keesler  as  an  example, 
the  table  shows  that  in  the  second  quarter  it  had  925  work  orders 
and  9,069  job  orders  available  for  completion  and  4,757  recurring 
work  actions  scheduled  for  accomplishment.  This  resulted  in 
Keesler  completing  8.76%  of  its  available  work  orders,  69-5%  of 
its  available  job  orders,  and  75.2%  of  scheduled  recurring  work 

i 

actions. 

Table  6  compares  the  efficiency  ratings  from  Analysis  #2  and 

* 

Augmented  Studies  #1  and  02.  Note  that  in  general  the  hQ  values 
for  each  base  do  not  vary  much  between  the  analyses  as 
illustrated  by  the  results  for  Keesler.  Augmenting  Analysis  #2 
with  three  additional  inputs  (AS1)  or  three  additional  outputs 
(AS2)  did  not  vary  Keesler's  efficiency  rating.  It's  second 
quarter  ratings  were  .945  (Analysis  02),  .946  (AS1),  and  .943 


>•  <*■ v  v,;t' 


Base 


Keesler 


Lowry 


Mather 


Keese 


Sheppard 


Vance 


Table  5 

Additional  Observed  Values  For  Augmented  Studies 

Input  & 


Output 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

AWO 

925 

743 

703 

AJO 

9069 

8644 

9238 

SRWA 

4757 

5251 

4910 

JCWO 

8.76 

10.36 

9-82 

JCJO 

69.  J 

75.5 

71.8 

JCRWA 

75.2 

79.7 

84.0 

AWO 

117 

1261 

1318 

AJO 

6074 

6064 

6229 

SRWA 

3113 

4317 

3898 

JCWO 

4.  42 

6.03 

5.84 

JCJO 

62.6 

65.7 

62.0 

JCRWA 

80.6 

107.5 

92.  1 

AWO 

725 

697 

683 

AJO 

6173 

4867 

6054 

SRWA 

1578 

1584 

2401 

JCWO 

9-38 

9.04 

7.61 

JCJO 

74.6 

68.7 

75.3 

JCRWA 

57.0 

62.2 

61.8 

AWO 

616 

578 

737 

AJO 

2907 

3194 

3324 

SRWA 

1691 

1539 

14335 

JCWO 

8.11 

9.34 

7.73 

JCJO 

78.9 

79-0 

77.7 

JCRWA 

81.7 

101.2 

96.2 

AWO 

1737 

1569 

1608 

AJO 

7449 

8970 

8697 

SRWA 

2403 

867 

2596 

JCWO 

7.77 

7.65 

8.58 

JCJO 

79.4 

86.3 

84.4 

JCRWA 

57.9 

103. 1 

97.3 

AWO 

1014 

1174 

1085 

AJO 

2283 

3161 

2472 

SRWA 

983 

1323 

1801 

JCWO 

13.02 

12.01 

12.72 

JCJO 

66.6 

53.8 

72.8 

JCKWA 

68.0 

82.7 

70.9 

Table  5  Continued 


Base 

Input  & 
Output 

2nd  Qtr 

3rd  Otr 

4th  Otr 

Williams 

AWO 

523 

468 

421 

AJO 

3986 

3908 

4741 

SHWA 

2711 

3337 

4035 

*CWO 

11.85 

12.82 

13-78 

%CJO 

72.6 

73.4 

75.7 

KCRWA 

61.4 

92.3 

83.6 

LEGEND 

AWO  -  Available  Work  Orders 

A JO  -  Available  Job  Orders 

SRWA  -  Scheduled  Recurring  Work  Actions 

%CWO  -  Percent  of  Work  Orders  Completed 

%CJO  -  Percent  of  Job  Orders  Completed 
1CRWA  -  Percent  of  Recurring  Work  Actions  Completed 


Table  6 

Comparison  Of  Efficiency  Ratings 
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Base 

Analysis 

2nd  Qtr 

3rd  Otr 

4th  Otr 

Keesler 

2 

.945 

•  994 

1.0 

AS1 

.946 

.993 

.999 

AS2 

.943 

1.0 

1.0 

Lowry 

2 

.690 

.968 

.791 

AS1 

.713 

.968 

.791 

AS2 

.809 

1.0 

.881 

Mather 

2 

.928 

.645 

.849 

AS1 

.938 

.725 

.894 

AS2 

1.0 

.898 

.972 

Reese 

2 

1.0 

.967 

.827 

AS1 

1.0 

1.0 

1 . 0 

AS2 

1.0 

1.0 

.968 

Sheppard 

2 

.893 

1.0 

1.0 

AS1 

.893 

1.0 

1 . 0 

AS2 

.943 

1.0 

1.0 

Vance 

2 

1.0 

1.0 

1.0 

AS1 

1.0 

1.0 

1.0 

AS2 

1.0 

1.0 

1.0 

Williams 

2 

.899 

1.0 

1.0 

AS1 

.966 

1.0 

1 . 0 

AS2 

.971 

1.0 

1.0 

XAS2).  hather  probably  could  be  considered  the  lone  exception  to 
these  "invariance"  findings.  Hence,  it  would  appear  from  this 
comparison  that  the  introduction  of  these  additional  inputs  or 
outputs  would  not  significantly  alter  the  results  of  our 
analyses . 

Two  Quarter  vs  Three  Quarter  Window  Analyses 

in  the  preceding  analyses,  we  turned  to  accomplishing  a 
series  of  window  type  analyses  using  data  from  three  fiscal 
quarters  in  order  to  increase  the  number  of  observations  and 
thereby  attain  a  more  reliable  efficiency  rating.  Thus,  we  are 
also  interested  in  the  sensitivity  of  the  results  of  the  window 
analyses  to  how  we  form  the  windows  used  in  the  analysis,  and 
hence  we  accomplished  a  second  series  of  window  analyses  using 

two  quarters  of  data  for  each  window. 

* 

Table  7  shows  the  hQ  values  for  this  new  series  of  window 

« 

analyses.  It  can  be  compared  with  Table  2  which  contains  the  hQ 
values  for  the  three-quarter  window  analyses.  As  can  be  seen 
from  reviewing  these  tables,  the  ratings  and  trends  in  both 
analyses  are  very  similar. 

For  example,  both  series  of  analyses  have  similar  efficiency 
ratings  for  Keesler.  Both  gave  a  100%  efficient  rating  for  the 
first  and  fifth  quarters.  The  other  quarters  were  also  very 
similar.  Note  the  fourth  quarter  where  the  two-quarter  window 
analyses  computed  ratings  of  1.0  and  .963  which  are  very  close 
(indeed  almost  identical)  with  the  ratings  of  1.0  and  .961  forjthe 
three-quarter  window  analyses  shown  in  Table  2. 


Table  7 


Efficiency  Ratings 
Window  Analysis  Using  Two  Quarters 


Base 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

Kees 1 er 

1.0 

1.0 

.963 

1.0 

.990 

1.0 

.963 

1.0 

Lowry 

.935 

.859 

.789 

1.0 

.  968 

.  792 

.768 

1.0 

Mather 

1.0 

.854 

.931 

.685 

.645 

.847 

.857 

.838 

Reese 

1.0 

.907 

1.0 

.983 

1.0 

.895 

.919 

1.0 

Sheppard 

1.0 

1.0 

.917 

1.0 

1.0 

1.0 

1.0 

1.0 

V  ance 

1.0 

1.0 

1.0 

1.0 

1.0 

1.0 

1.0 

1.0 

Willi ams 

1.0 

1.0 

.964 


Willi ams 


1.0 


1.0 


1.0 


In  bather's  case  both  series  of  analyses  show  its 
performance  becoming  progressively  worse  through  the  third 


quarter  of  FY  83  and  then  showing  some  improvement.  This  is  in 
addition  to  having  very  similar  ratings  from  the  two  types  of 
analyses . 

Ratings  Sensitivity  Summary 

To  further  check  the  stability  of  our  evaluations,  we  tested 
their  sensitivity  to  the  addition  of  different  inputs  and  out¬ 
puts  and  changes  in  the  number  of  DMUs  used  in  the  analysis.  We 
found  that,  in  general,  the  ratings  were  unaffected  by  the 
introduction  of  the  inputs  and  outputs  we  eliminated  in  the  first 
part  of  this  chapter.  The  results  also  indicated  that  using  a 
two-quarter  window  analysis  versus  a  three-quarter  window 
analysis  will  not  significantly  alter  the  efficiency  ratings. 
Hence  the  results  from  our  previous  analyses  continue  to  stand 
with  these  additional  validations  —  i.e.,  stability  of  results 
not  affected  by  variations  either  in  the  number  of  inputs  and 
outputs  or  in  the  number  of  DMUs. 


CHAPTER  3 

SUHHARY  AND  CONCLUSIONS 


In  this  research  we  reviewed  and  discussed  efficiency 
ratings  and  trends  for  Air  Force  bases  in  the  Air  Training 
Command.  The  research  is  built  around  an  efficiency  measurement 
methodology  developed  by  Charnes,  Cooper,  and  Rhodes  [6]  and 
others  called  Data  Envelopment  Analysis  (DEA).  DEA  provides  a 
relative  measure  of  efficiency  for  organizations  that  have 
multiple  inputs  and  multiple  outputs  and  for  which  an  a  priori 
production  function  is  not  available. 

we  initiated  this  study  by  applying  DEA  to  annual  data  from 
ATC  bases.  As  expected,  almost  all  bases  were  rated  100% 
efficient.  This  was  due  to  having  an  insufficient  number  of 
observations  to  effect  a  Data  Envelopment  Analysis  which  would 
discriminate  between  efficient  and  less  efficient  bases. 

We  increased  the  available  number  of  observations  via  a 
three-quarter  window  analysis  and  a  joint  analysis  using  data 
from  two  commands.  This  allowed  us  to  test  the  stability  and 
validity  of  the  measures  obtained  from  evaluating  the  bases  using 
solely  annual  data.  It  also  provided  supplemental  information  on 
trends  and  cycles. 

Using  the  three-quarter  window  analysis  in  conjunction  with 
the  analysis  of  annual  data  we  can  come  to  the  following 
conclusions  on  the  operational  efficiency  of  the  bases  included 
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(1)  Keesler,  Sheppard,  Vance,  and  Williams,  seem  to  be 
operating  efficiently. 

(2)  Lowry  and  Mather  clearly  have  the  most  inefficient 
operations . 

(3)  Reese’s  results  are  not  quite  as  clear  cut  but  they  do 
indicate  that  Reese  has  some  operational  inefficiencies. 

However,  it  does  not  appear  to  be  as  inefficient  as  either  Lowry 
or  Mather. 

(4)  Our  analysis  also  provided  indications  that  Vance  was 
rated  efficient  because  of  special  operational  character istics 
which  are  different  from  the  civil  engineering  operations  of  the 
other  bases  —  possibly  its  contract  operations. 

(5)  Finally,  we  noted  that  there  was  a  concentration  of 
efficient  operations  in  the  first  quarter  of  the  fiscal  year 
while  a  majority  of  the  second  quarter  operations  were 
inefficient.  This  supports  the  hypothesis  that  a  lack  of  an 
appropriations  causes  significant  planning  problems  in  the  use  of 
resources . 

Recall  that  these  ratings  are  relative  and  dependent  on  the 
DMUs  and  inputs/outputs  included  in  the  analysis.  Thus,  we  also 
tested  the  sensitivity  of  our  findings  to  the  addition  of  new 
inputs  and  outputs  and  to  how  we  formed  the  windows  for  the 
window  analysis.  We  found  that  these  changes  had  little  effect 
on  the  efficiency  ratings. 

Finally,  it  is  important  to  note  that  these  findings  are  not 
to  be  used  solely  as  an  end  in  themselves  but  also  as  a  means  to 
an  end.  That  end  being  efficient  operations.  This  information 
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‘should  be  used  as  guides  for  further  investigation  into  how 
anjarganization  can  become  operationally  efficient.  Information 
supplied  via  the  identification  of  the  amount  of  inefficiency  in 
each  input  and  output  and  the  bases  in  the  inefficient  unit’s 
comparison  set  can  be  used  for  this  purpose. 


APPENDIX  A 


SUPPLEMENTAL  TABLES 

This  appendix  contains  supplemental  tables  reporting  the 
data  used  in  and  the  results  of  the  efficiency  evaluations  for 
Air  Training  Command  and  MAJCOM  #2. 


TABLES  A. 1  AND  A. 2 

Tables  A. 1  and  A. 2  report  the  observed  input  and  output 
values  for  ATC  and  MAJCOM  #2  used  in  this  research.  They  are  the 
actual  amount  of  resources  consumed  by  a  base  in  producing  its 
output  over  the  time  period  shown. 

The  annual  values  shown  in  column  3  of  Table  A. 1  are  for 
Fiscal  Year  (=FY)  83*  For  example,  Keesler  used  inputs  of  38 
passenger  carrying  vehicles  (VEH),  $3,247,600  worth  of  supplies 
and  equipment  (DOL),  and  453,951  direct  labor  hours  (LAB  HR)  in 
FY  83  to  produce  308  completed  work  orders  (CWO),  25,451 
completed  job  orders  (CJO),  15,962  completed  recurring  work 
actions  (CRWA),  and  3,629  delinquent  job  orders  (DJO)  as  outputs 
for  the  year.  Replacing  the  annual  accomplishments  and  efforts 
with  their  reported  quarterly  values  we  have,  for  example, 

Keesler  using  38  vehicles,  $779,240  and  106,022  direct  labor 
hours  to  complete  81  work  orders,  5,983  job  orders,  and  4,072 
recurring  work  actions  with  913  job  orders  being  delinquent  in 
the  first  quarter  of  FY  83* 
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The  other  cells  are  similarly  interpreted  but,  of  course, 
some  treatment  of  these  data  may  be  needed  for  DEA.  For  example, 
recall  from  our  earlier  discussion  in  Chapter  1  that  we  need  to 
use  the  reciprocal  of  the  number  of  delinquent  job  orders  as  our 
measure  to  reflect  the  fact  that  increases  in  delinquent  job 
orders  are  not  desirable.  Thus,  the  913  delinquent  job  orders 
reported  by  Keesler  for  the  first  quarter  in  column  4  of  Table 
A. 1  would  be  replaced  by  1/913  or  .00110  in  DEA  development. 
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Table  A.i 

Observed  Inputs  and  Outputs  for  ATC 
Fiscal  Year  19H3  Data 


(1)  U)  (3)  (4)  (5)  (6)  (7)  (8) 

Input/ 


Base  Output 

Annual 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

Klees  ler  VEH 

38 

IS 

38 

38 

38 

38 

DDL  3 

,247,600 

779,240 

810,450 

830,250 

827,750 

616,015 

LAB  HR 

453,951 

106,022 

115,083 

117,901 

114,945 

98,247 

CWO 

308 

81 

81 

77 

69 

71 

CJO 

25,451 

5,983 

6,307 

6,530 

6,631 

6,882 

CRWA 

15,962 

4,072 

3,578 

4,186 

4,126 

4,443 

UJG 

3,629 

913 

977 

912 

827 

283 

Lowry  VEH 

33 

33 

33 

33 

33 

33 

DOL  2 

,507,700 

576,242 

640,971 

640,000 

650,500 

536,997 

LAB  HR 

250,169 

57,304 

62,821 

62,878 

67,166 

57,731 

CWO 

294 

89 

52 

76 

77 

56 

CJO 

14,b97 

3,045 

3,804 

3,986 

3,862 

4,136 

CRWA 

13,517 

2,780 

2,508 

4,639 

3,590 

9,028 

OoO 

2,705 

982 

599 

355 

769 

1,038 

LEGEND 

Inputs 

VEH  -  Passenger  Carrying  Vehicles 
DDL  -  Supply  and  Equipment  Funding 
LAB  HR  -  Available  Direct  Labor  Hours 

Outputs 

CWO  -  Completed  Work  Orders 
CJO  -  Completed  Job  Orders 
CRWA  -  Completed  Recurring  Work  Actions 
UJG  -  Delinquent  Job  Orders 


I 


I 

i 


i 

i 


i 

i 


1 


i 
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Ui 

Base 

(2) 

Input/ 

Output 

(3) 

Annual 

Table  A.l 

(4) 

1st  Qtr 

Continued 

(5) 

2nd  Qtr 

(6) 

3rd  Qtr 

(7) 

4th  Qtr 

(8) 

5th  Qtr 

Mather 

“VEH - 

29 

29 

29 

29 

29 

29 

DOL  3 

,491,000 

910,270 

875,762 

780,0000 

925,000 

501,000 

LAB  HR 

265,344 

62,219 

66,201 

65,845 

71,079 

68,623 

CWO 

251 

68 

68 

63 

52 

28 

CJO 

18,940 

6,434 

4,608 

3,341 

4,557 

4,223 

CRWA 

5,222 

1,851 

900 

986 

1,485 

3,294 

UJO 

2,306 

590 

246 

601 

869 

538 

Reese 

VEH 

24 

24 

24 

24 

24 

24 

DOL  1 

,846,700 

450,250 

396,450 

525,000 

475,000 

237,511 

LAB  HR 

150,677 

31,322 

34,315 

41,269 

43,772 

34,243 

CWU 

215 

54 

50 

54 

57 

58 

CJO 

9,341 

1,943 

2,293 

2,522 

2,583 

2,515 

CRWA 

5,672 

1,352 

1,382 

1,558 

1,380 

1,596 

DJO 

394 

52 

82 

99 

161 

224 

LEGEND 

Inputs 

VEH  -  Passenger  Carrying  Vehicles 
DOL  -  Supply  and  Equipment  Funding 
LAB  HR  -  Available  Direct  Labor  Hours 

Outputs 

CWO  -  Completed  Work  Orders 
CJO  -  Completed  Job  Orders 
CRwA  -  Completed  Recurring  Work  Actions 
UJO  -  Delinquent  Job  Orders 
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Table  A.l  Continued 


(1) 

base 

U) 

Input/ 

Output 

(3) 

Annual 

(4) 

1st  Qtr 

(5) 

2nd  Qtr 

(6) 

3rd  Qtr 

(7) 

4th  Qtr 

(8) 

5th  Qtr 

Sheppard  VEH 

40 

40 

40 

DOL  3 

,ib3,300 

810,070 

791,683 

775,000 

776,550 

575,368 

LA6  HR 

368,356 

85,692 

91,261 

96,540 

94,864 

85,552 

CWO 

513 

120 

135 

120 

138 

121 

CJO 

27,141 

6,147 

5,913 

7,743 

7,338 

6,039 

CKWA 

6,748 

1,936 

1,391 

894 

2,527 

2,207 

DJO 

1,860 

495 

468 

537 

360 

364 

Vance 

VEH 

23 

23 

23 

23 

23 

23 

DOL  1 

,691,700 

402,964 

391,242 

456,270 

441,250 

329,206 

LAB  HR 

197,939 

45,675 

47,141 

51,552 

53,572 

47,515 

CWO 

539 

128 

132 

141 

138 

134 

CJO 

6,790 

1,770 

1,520 

1,700 

1,800 

1,708 

CRWA 

4,267 

1,228 

668 

1,094 

1,277 

1,536 

DJO 

941 

253 

297 

246 

145 

110 

LEGEND 

Inputs 

VEH  -  Passenger  Carrying  Vehicles 
DOL  -  Supply  and  Equipment  Funding 
LAB  HR  -  Available  Direct  Labor  Hours 

Outputs 

CWO  -  Completed  Work  Orders 
CJO  -  Completed  Job  Orders 
CRWA  -  Completed  Recurring  Work  Actions 
DJO  -  Delinquent  Job  Orders 
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Table  A.l  Continued 


(1)  (2) 
Input/ 
Base  Output 

(3) 

Annual 

(4) 

1st  Qtr 

(5) 

2nd  Qtr 

(6) 

3rd  Qtr 

(7) 

4th  Qtr 

(8) 

5th  Qtr 

Williams m - 

- 22 

22 

22 

22 

22 

22“ 

DGL  1 

,848,600 

471,983 

461,298 

465,070 

450,291 

376,362 

LAB  HR 

126,060 

28,484 

35,013 

29,453 

33,110 

31,414 

CWO 

242 

62 

62 

60 

58 

30 

CJO 

11,940 

2,589 

2,893 

2,869 

3,589 

3,214 

CRWA 

10,686 

2,767 

1,665 

3,081 

3,373 

3,650 

UJO 

1,323 

431 

302 

304 

286 

346 

LEGEND 

Inputs 

VEH  -  Passenger  Carrying  Vehicles 
DOL  -  Supply  and  Equipment  Funding 
LAB  HR  -  Available  Direct  Labor  Hours 

Outputs 

CWO  -  Completed  Work  Orders 
COO  -  Completed  Job  Orders 
CRWA  -  Completed  Recurring  Work  Actions 
0J0  -  Delinquent  Job  Orders 


I 


Table  A. 2  Continued 


Observed  Inputs  and  Outputs  £or  TAC 
Fiscal  Year  1983  Data 


Input/ 


Base 

Output 

Annual 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

Luke 

VEH 

44 

44 

44 

44 

44 

44 

DOL  2 

,444,700 

461,100 

684,600 

474,000 

825,000 

437,600 

LAB  HR 

265,866 

57,645 

70,334 

69,028 

68,859 

61,913 

CWO 

197 

52 

53 

61 

31 

36 

CJO 

16,878 

3,811 

4,180 

4,191 

4,696 

3,880 

CRWA 

12,860 

2,554 

2,277 

4,756 

3,273 

2,863 

DJO 

2,405 

627 

612 

649 

517 

592 

Howard 

VEH 

42 

42 

42 

42 

42 

DOL  2 

,887,300 

599,800 

780,600 

748,800 

758,100 

LAB  HR 

338,611 

74,158 

81 ,969 

87,717 

758,100 

CWO 

135 

32 

46 

14 

43 

CJO 

30,130 

7,914 

7,276 

7,094 

7,846 

CRWA 

3,492 

873 

873 

873 

873 

DJO 

4,951 

1 ,694 

791 

993 

1  ,473 

LBGBND 

Inputs 

VEH 
DOL 
LAB  HR 


Passenger  Carrying  Vehicles 
Supply  and  Equipment  Funding 
Available  Direct  Labor  Hours 


Outputs 

CWO 

CJO 

CRWA 


Completed  Work  Orders 
Completed  Job  Orders 
Completed  Recurring  Work  Actions 


DJO  -  Delinquent  Job  Orders 
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Table  A. 2  Continued 


Input/ 


Base  Output 

Annual 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

Langley  Vfer! 

HHHBIUJI 

UMIlMj 

■gjHWMripJB 

■HWlV| 

DOL 

4,304,100 

934,500  1,387,300 

1,155,500 

826,800 

LAB  HR 

526,896 

123,463 

127,217 

141,237 

134,979 

CWO 

162 

40 

42 

34 

44 

CJO 

29,690 

8,130 

6,870 

6,910 

7,780 

CRWA 

11,361 

2,687 

3,068 

3,3390 

2,486 

DJO 

21,806 

5,698 

5,604 

6,441 

4,063 

George  VEH 

44 

44 

44 

44 

44 

44 

DOL 

2,302,400 

693,200 

667,300 

554,600 

387,300 

850,500 

LAB  HR 

237,136 

56,126 

62,055 

65,242 

53,713 

44,220 

CWO 

327 

65 

74 

72 

116 

97 

CJO 

30,110 

7,772 

7,515 

6,686 

8,137 

6,180 

CRWA 

7,075 

2,706 

1,314 

2,230 

825 

7,158 

DJO 

3,523 

1,104 

943 

780 

696 

1,323 

LEGEND 

Inputs 

VEH  -  Passenger  Carrying  Vehicles 
DOL  -  Supply  and  Equipment  Funding 
LAB  HR  -  Available  Direct  Labor  Hours 


Outputs 

CWO  -  Completed  Work  Orders 
CJO  -  Completed  Job  Orders 
CRWA  -  Completed  Recurring  Work  Actions 
DJO  -  Delinquent  Job  Orders 
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Table  A .2  Continued 


Input/ 


Base  Output 

Annual 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

Moody  VEH 

19 

17 

.  17 

20 

23 

23 

DOL 

1,787,800 

401,500 

432,800 

520,000 

433,500 

339,300 

LAB  HR 

210,869 

44,674 

53,978 

57,680 

54,537 

53,lb2 

CWO 

193 

15 

71 

35 

72 

74 

CJO 

12,348 

2,815 

3,133 

3,255 

3,145 

3,672 

CRWA 

9,244 

2,668 

2,493 

2,086 

1,997 

1,460 

DJO 

2,465 

739 

658 

396 

672 

673 

Shaw  VEH 

71 

71 

71 

71 

71 

71 

DOL 

3,823,800 

957,300 

927,400 

774,700 

1,164,400 

783,200 

LAB  HR 

254,816 

61,978 

61,537 

64,539 

66, 7o2 

65,067 

CWO 

122 

21 

31 

26 

44 

38 

CJO 

15,593 

3,608 

3,563 

3,859 

4,563 

4,818 

CRWA 

5,981 

1,509 

2,295 

1,248 

929 

2,487 

DJO 

1,387 

316 

423 

444 

204 

7 

LEGEND 

1 nputs 

VEH  -  Passenger  Carrying  Vehicles 
DOL  -  Supply  and  Equipment  Funding 
LAB  HR  -  Available  Direct  Labor  Hours 


Outputs 

(TWO  -  Completed  Work  Orders 
CJO  -  Completed  Job  Orders 
CRWA  -  Completed  Recurring  Work  Actions 
DJO  -  Delinquent  Job  Orders 
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Table  A. 2  Continued 


Base 

Input/ 

Output 

Annual 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

Myrtle 

VEH 

14 

14 

14 

14 

14 

14 

Beach 

DOL  2, 

127,600 

427,200 

492,300 

554,500 

653,600 

419,200 

LAB  HR 

176,146 

40,209 

40,205 

46,132 

49,600 

46,734 

CWO 

579 

183 

123 

94 

179 

109 

CJO 

17,060 

3,160 

4,408 

4,501 

4,991 

3,437 

CRWA 

9,756 

2,692 

837 

3,171 

3,056 

2,705 

DJO 

3,724 

960 

835 

1,000 

929 

728 

Holloman 

i  VEH 

35 

39 

37 

32 

32 

32 

DOL  3 , 

668,100 

674,500 

953,600 

1,021,400 

1,018,600 

814,200 

LAB  HR 

306,498 

77,339 

81,422 

74,216 

73,520 

63,788 

CWO 

190 

43 

58 

65 

24 

34 

CJO 

14,800 

3,012 

3,961 

3,588 

4,239 

3,588 

CRWA 

10,546 

3,162 

1,584 

3,121 

2,679 

2,854 

DJO 

10,464 

3,305 

2,988 

2,160 

2,011 

1,596 

Passenger  Carrying  Vehicles 
Supply  and  Equipment  Funding 
Available  Direct  Labor  Hours 


Completed  Work  Orders 
Completed  Job  Orders 
Completed  Recurring  Work  Actions 
Delinquent  Job  Orders 


LEGEND 

Inputs 

VEH 
DOL 
LAB  HR 


Outputs 

CWO 

CJO 

CRWA 

DJO 
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Table  A. 2  Continued 


Input/ 


Base  Output 

Annual 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

Bergstrom  VEH 

38 

38 

38 

38 

38 

38 

DOL  2 

,406,900 

382,200 

619,700 

593,500 

811 ,500 

665,400 

LAB  HR 

237,951 

55,448 

59,509 

63,822 

59,172 

64,008 

CWO 

171 

60 

36 

39 

36 

58 

CJO 

10,773 

3,021 

2,737 

2,579 

2,436 

2,565 

CRWA 

8,453 

1  ,938 

1  ,635 

2,058 

2,822 

2,931 

DJO 

1  ,928 

473 

415 

439 

601 

1  ,028 

LEGEND 

Inputs 

VEH 
DOL 
LAB  HR 


Passenger  Carrying  Vehicles 
Supply  and  Equipment  Funding 
Available  Direct  Labor  Hours 


Outputs 

CWO  -  Completed  Work  Orders 
CJO  -  Completed  Job  Orders 
CRWA  -  Completed  Recurring  Work  Actions 
DJO  -  Delinquent  Job  Orders 


NOTE:  Recall  from  our  earlier  discussion  that  we  need  to 

use  the  reciprocal  of  the  DJO  measure.  Therefore,  the 
actual  value  used  in  effecting  DEA  would  be,  for  example, 
1/431  or  .00232  for  base  G,  first  quarter. 


Table  A. 3 


Table  A. 3  displays  the  two  factors,  0*  and  optimal  slack 
variable  values,  which  compose  the  efficiency  rating  for  the 
three  series  of  window  analyses  undertaken  for  ATC.  Analysis  #1 
consists  of  data  from  the  first,  second,  and  third  quarters  for 
each  base.  Data  from  the  second,  third,  and  fourth  quarters  form 
the  windows  in  Analysis  if2  while  Analysis  if 3  involves  data  from 
the  third,  fourth,  and  fifth  quarters. 

The  0*  values  listed  for  Analyses  if  1,  if 2 ,  and  if 3  in  columns 
4  through  8  are  the  same  as  those  reported  in  Table  2.  They  are 

repeated  here  so  as  to  show  both  parts  of  the  efficiency  analysis 

« 

in  one  place.  Note  that  0*  =  hQ  when  the  non-Archimedean 
conditions  are  fulfilled. 

The  slack  variable  values  which  appear  under  the  efficiency 
ratings  in  each  column  correspond  to  the  input  or  output  measure 
with  which  the  slack  variable  is  associated  for  each  analysis 
shown  in  column  3« 

We  will  use  Keesler  to  illustrate  how  to  read  this  table. 

In  Analysis  if  1 ,  Keesler's  second  quarter  received  an  efficiency 

* 

rating,  0*  =  h^,  of  .963  with  zero  values  for  all  slack  variables 
except  for  LAB  HR  which  equaled  4,791  hours  and  DJO  which  equaled 
3.3  Keesler's  second  quarter  is  also  a  DMU  in  Analysis  if 2.  From 
this  new  reference  set,  Keesler,  second  quarter  had  0*  =  .945 
with  positive  slack  variable  values  of  DOL  =  $46,971 >  LAB  HR  = 
41,339  hours,  CWO  r  19  work  orders,  and  DJO  =  30.4. 


As  discussed  in  Chapter  1  when  we  described  the  delinquent 
job  order  output  measure,  we  use  a  reciprocal  of  the  number  of 
delinquent  job  orders  as  the  actual  DJO  measure  when  effecting 
DEA.  Hence,  the  DJtf  slack  variable  values  shown  in  this  table 
have  little  intuitive  meaning.  However,  they  do  have  value  in 
that  they  are  an  important  factor  in  determining  Most  Productive 
Scale  Size. 

Finally,  the  *  in  column  indicates  that  the  slack  variable 
had  a  value  of  zero  for  that  analysis. 


rsw-j) 
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Table  A.3 

Efficiency  Measures  For  ATC 


Window  Analyses 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

Base 

Measures 

Analysis 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

Keesler 

-  h0 

#1 

1.0 

.963 

1.0 

#2 

.945 

.994 

1.0 

#3 

.957 

.961 

1.0 

Slack 

Values 

, 

VEH 

#1 

* 

* 

★ 

#2 

* 

★ 

* 

#3 

★ 

★ 

★ 

DOL 

#1 

★ 

★ 

★ 

#2 

46971 

65486 

* 

n 

176566 

198656 

* 

LAB  HR 

#1 

•k 

4791 

★ 

n 

41339 

49199 

* 

#3 

23762 

16303 

* 

CWO 

#i 

* 

* 

★ 

#2 

19 

27 

* 

#3 

* 

1 

* 

Cdu 

#1 

* 

* 

* 

#2 

* 

* 

* 

#3 

* 

* 

* 

CRWA 

#1 

* 

★ 

* 

#2 

★ 

* 

* 

#3 

* 

* 

* 

DdU 

#1 

* 

3.3 

* 

#2 

30.4 

35.3 

* 

#3  ' 

24.4 

21.1 

★ 
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Table  A. 3  Continued 


U) 

Base 

Lowry 


(2) 

Measures 

(3) 

Analysis 

(4) 

1st  Qtr 

(5) 

2nd  Utr 

(6) 

3rd  Qtr 

(7) 

4th  Qtr 

(8) 

5th  Qtr 

b*  •  h* 

#1 

.874 

.789 

1.0 

#2 

.690 

.968 

.791 

#3 

.828 

.768 

1.0 

Slack 

Values 

VEH 

#1 

2 

1 

* 

#2 

1 

2 

1 

* 

* 

* 

DOL 

#1 

* 

* 

* 

#2 

* 

* 

* 

#3 

* 

* 

* 

LA6  HR 

11 

* 

* 

* 

#2 

1599 

15307 

12524 

#3 

7039 

7116 

* 

Ck»0 

#1 

* 

16 

* 

#2 

14 

4 

* 

#3 

* 

* 

* 

Cju 

#1 

* 

* 

* 

#2 

• 

950 

16 

#3 

* 

* 

* 

CKWA 

*1 

* 

* 

* 

#2 

* 

* 

* 

#3 

* 

* 

* 

DJd 

#1 

20.4 

5.7 

* 

#2 

9.7 

19.9 

31.6 

#3 

11.3 

24.4 

* 

tm  iMiMirmnHivnyr 
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(1) 

Base 

Mather 


Table  A. 3  Continued 


(2) 

Measures 

(3) 

Analysis 

(4) 

1st  Qtr 

(5) 

2nd  Qtr 

(6) 

3rd  Qtr 

(7) 

4th  Qtr 

«*  =  h* 

#1 

1.0 

.849 

.640 

0 

#2 

.928 

.645 

.849 

#3 

.645 

.836 

Slack 

Values 

VEH 

*1 

* 

★ 

* 

#2 

★ 

* 

* 

#3 

* 

* 

00L 

#1 

★ 

49166 

* 

#2 

294258 

138250 

290304 

#3 

137983 

331132 

LAB  HR 

#1 

* 

5716 

1295 

#2 

2940 

* 

* 

#3 

* 

* 

CWO 

#1 

* 

* 

■k 

*2 

7 

★ 

1 

#3 

* 

1 

CJO 

#1 

* 

* 

* 

#2 

* 

* 

* 

#3 

* 

* 

CKWA 

#1 

* 

602 

25 

#2 

* 

351 

* 

#3 

361 

* 

UJU 

#1 

★ 

* 

* 

#2 

* 

★ 

2.6 

#3 

* 

4.7 

(8) 

5th  Qtr 


.838 


11777 


20 


4.8 
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(1) 

Base 


Reese 


Table  A. 3  Continued 


(2) 

13) 

(4) 

(5) 

(6) 

(7) 

(8) 

Measures 

Analysis 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

0*  *  h* 

n 

1.0 

.887 

.875 

u 

#2 

1.0 

.967 

.827 

#3 

1.0 

.850 

1.0 

Slack 

Values 

VEH 

#1 

* 

* 

* 

#2 

* 

* 

* 

#3 

* 

* 

* 

UUL 

#1 

* 

★ 

23622 

#2 

* 

67045 

11997 

#3 

* 

* 

* 

LAB  HR 

#1 

* 

* 

3800 

#2 

* 

4368 

281 

#3 

★ 

* 

* 

CWO 

#1 

* 

* 

* 

#2 

* 

* 

* 

#3 

* 

* 

* 

CJO 

#1 

* 

* 

* 

#2 

* 

* 

* 

#3 

* 

* 

* 

CKWA 

#1 

* 

93 

* 

#2 

* 

* 

★ 

#3 

* 

531 

* 

DJO 

#1 

* 

* 

* 

#2 

* 

* 

* 

#3 

* 

* 

* 
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Table  A.3  Continued 


(1)  U) 

Base  Measures 

(3) 

Analysis 

(4) 

1st  Qtr 

(5) 

2nd  Qtr 

(6) 

3rd  Qtr 

(7) 

4th  Qtr 

* 

Sheppard  8*  *  hA 

#1 

.919 

.916 

1.0 

U 

#2 

.893 

1.0 

1.0 

#3 

1.0 

1.0 

Slack 

Values 

VEH 

#1 

* 

★ 

* 

#2 

* 

★ 

* 

#3 

* 

* 

DDL 

#1 

* 

* 

* 

*2 

8194 

* 

* 

#3 

* 

* 

LAB  HR 

#1 

* 

* 

* 

#2 

* 

* 

* 

#3 

* 

* 

CUO 

#1 

* 

* 

* 

#2 

* 

* 

* 

#3 

* 

* 

CJO 

#1 

* 

* 

* 

#2 

★ 

* 

* 

#3 

* 

* 

CRWA 

#1 

* 

* 

* 

#2 

1046 

* 

* 

#3 

* 

* 

DJO 

#1 

10.8 

9.7 

* 

#2 

11.7 

* 

* 

#3 

* 

* 

Vance  6*  *  tu 

#1 

1.0 

1.0 

1.0 

U 

#2 

1.0 

1.0 

1.0 

#3 

1.0 

1.0 

Slack 

Values 


(8) 

5th  Qtr 


1.0 


* 


* 


★ 


* 


* 


★ 


* 


1.0 


All  slack  values  equal  zero  for  all  windows 


Table  A.3  Continued 


1)  (2) 
ise  Measures 

(3) 

Analysis 

(4) 

1st  Qtr 

(5) 

2nd  Qtr 

(6) 

3rd  Qtr 

* 

illiams  8*  *  hn 

-  #1 

1.0 

.982 

1.0 

u 

#2 

.899 

1.0 

#3 

1.0 

Slack 

Values 

VEH 

#1 

* 

* 

* 

#2 

* 

* 

#3 

★ 

OUL 

fl 

* 

* 

* 

#2 

10585 

* 

#3 

LAB  HR 

n 

* 

* 

* 

#2 

* 

* 

#3 

* 

CWO 

#1 

* 

* 

* 

#2 

* 

* 

#3 

* 

CJO 

#1 

★ 

* 

* 

#2 

* 

* 

#3 

* 

CRWA 

#1 

* 

599 

* 

#2 

1044 

* 

#3 

* 

DjO 

#1 

* 

* 

* 

#2 

* 

* 

#3 

* 

72 


v 

Table  A. 4 

Table  A. 4  exhibits  the  8*  and  optimum  slack  variable  values 
resulting  from  the  window  analyses  of  the  combined  ATC  and  MAJCOM 
02  data.  It  uses  the  same  format  as  Table  A. 3  except  that  the 
three  window  analyses  are  referred  to  as  Analyses  #4,  05,  and  #6. 
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Base 

Keesler 


Table  A. 4 

Efficiency  Measures  For  ATC 
Joint  Window  Analyses 


Measures 

Analysis 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

e*  =  h! 

#4 

.847 

.821 

.863 

u 

#5 

.769 

.825 

.829 

#6 

.761 

.769 

955 

Slack 

• 

Values 

VEH 

#4 

* 

* 

* 

#5 

* 

* 

* 

#6 

* 

* 

* 

DGL 

#4 

* 

★ 

* 

#5 

★ 

★ 

* 

#6 

* 

* 

* 

LAB  HR 

#4 

22672 

29295 

31122 

#5 

26608 

29094 

2/087 

#6 

2B902 

27071 

31664 

CMC 

#4 

21 

22 

34 

#5 

33 

43 

58 

#6 

39 

49 

39 

COO 

#4 

* 

* 

* 

#5 

* 

* 

* 

#6 

* 

* 

* 

CKWA 

#4 

* 

* 

* 

#5 

* 

* 

* 

#6 

* 

* 

* 

OJU 

#4 

3.4 

3.4 

4.0 

IS 

3.9 

4.4 

3.3 

#6 

2.5 

1.6 

* 

-.*,*.*,  V.'t 
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Base 

Lowry 


Table  A.4  Continued 


Measures 

Analysis 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

8*  a  hn 

#4 

.670 

.656 

.999 

o 

#5 

.636 

.955 

.753 

#6 

.696 

.614 

1.0 

Slack 

Values 

VEH 

#4 

* 

* 

* 

#5 

* 

* 

* 

#6 

* 

* 

* 

DUL 

#4 

* 

* 

* 

#5 

* 

* 

if 

#6 

* 

it 

* 

LAB  HR 

#4 

* 

* 

* 

#5 

* 

8136 

1197 

#b 

* 

2053 

* 

CWG 

#4 

* 

8 

98 

#5 

18 

6 

* 

#6 

* 

* 

* 

CJO 

#4 

* 

* 

737 

#5 

* 

* 

* 

#6 

* 

* 

* 

CRWA 

#4 

* 

* 

* 

#5 

* 

* 

* 

#6 

* 

* 

* 

DJO 

#4 

#5 

* 

10.2 

3.3 

* 

#6 

* 

* 

* 
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Base 

Mather 


Table  A.4  Continued 


Measures 

Analysis 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

b*  =  h* 

#4 

.892 

.650 

.466 

o 

#5 

.672 

.459 

.567 

#6 

.445 

.571 

.776 

Slack 

Values 

VEH 

#4 

★ 

★ 

* 

#5 

★ 

* 

* 

#6 

* 

* 

* 

DOL 

#4 

131959 

39917 

* 

#5 

47417 

3873 

51455 

#6 

8939 

49265 

* 

LAb  HR 

#4 

* 

★ 

* 

#5 

* 

* 

k 

#6 

k 

k 

12963 

CWO 

#4 

67 

49 

9 

#5 

53 

25 

73 

#6 

30 

81 

40 

CJU 

#4 

* 

* 

* 

#5 

* 

k 

k 

#6 

* 

* 

k 

CRWA 

#4 

* 

220 

* 

#5 

179 

* 

* 

#6 

* 

581 

★ 

0  JO 

#4 

* 

★ 

★ 

#5 

* 

* 

* 

#6 

k 

* 

* 

76 


Table  A.4  Continued 


Base 


Reese 


Measures 

Analysis 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

e*  = 

#4 

1.0 

.838 

.740 

(J 

#5 

1.0 

.897 

.757 

»6 

.579 

.547 

.928 

Slack 

Values 

VEH 

#4 

★ 

* 

* 

#5 

* 

* 

* 

#6 

* 

* 

* 

UOL 

#4 

★ 

14213 

10107 

#5 

* 

20671 

2889 

#6 

* 

★ 

* 

LAB  HR 

#4 

★ 

★ 

* 

#5 

* 

2423 

* 

#6 

* 

31 

3475 

CWO 

#4 

* 

* 

3 

#5 

* 

5 

* 

#6 

3 

* 

* 

CJO 

#4 

* 

* 

* 

#5 

* 

* 

* 

#6 

* 

■k 

* 

CRWA 

#4 

* 

* 

* 

#5 

* 

* 

* 

#6 

* 

* 

* 

0J0 

#4 

* 

* 

* 

#5 

* 

* 

■k 

#6 

* 

* 

* 

7? 


Table  A.4  Continued 


Base  Measures 

Analysis 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

★ 

Sheppard  b*  =  h,. 

#4 

.741 

.742 

.918 

#5 

.651 

.819 

.825 

#6 

.833 

.797 

.777 

Slack 

Values 

VEH 

* 

* 

* 

#5 

* 

* 

* 

#6 

* 

* 

* 

00L 

#4 

* 

* 

* 

#5 

* 

★ 

* 

#6 

* 

* 

* 

LAB  HR 

#4 

981 

5499 

13587 

#5 

10645 

17481 

16041 

#6 

18250 

15676 

11515 

CtoU 

#4 

* 

* 

* 

#5 

* 

47 

* 

#6 

62 

22 

* 

Cju 

#4 

* 

* 

* 

#5 

* 

* 

* 

*6 

★ 

* 

* 

CRWA 

#4 

* 

708 

703 

#5 

112 

265 

* 

#6 

1590 

* 

* 

0J0 

#4 

* 

* 

* 

#5 

* 

* 

* 

16 

* 

* 

* 

I 


78 


Base 

Vance 


Table  A.4  Continued 


Measures 

Analysis 

1st  Qtr 

2nd  Qtr 

3rd  Qtr 

4th  Qtr 

5th  Qtr 

b*  =  h* 

#4 

.855 

.883 

.824 

u 

#5 

1.0 

.966 

1.0 

#6 

.893 

.873 

1.0 

Slack 

Values 

, 

VEH 

#4 

7 

7 

5 

#5 

* 

★ 

* 

#6 

1 

* 

* 

DOL 

#4 

* 

* 

* 

#5 

★ 

★ 

* 

#6 

* 

* 

* 

LAB  HR 

#4 

7647 

9947 

8123 

#5 

* 

69 

* 

#b 

* 

823 

* 

CWU 

#4 

★ 

* 

* 

#5 

* 

* 

* 

#6 

* 

* 

* 

CuO 

#4 

612 

899 

909 

#5 

* 

330 

* 

#6 

871 

544 

* 

CRWA 

#4 

750 

1351 

1076 

#5 

* 

29 

* 

#6 

809 

522 

* 

DJO 

#4 

* 

* 

★ 

#5 

* 

* 

* 

#6 

23.3 

.7 

* 

79 


Table  A. 4  Continued 


Base 

Williams 


Measures  Analysis  1st  Qtr  2nd  Qtr  3rd  Qtr  4th  Qtr  5th  Qtr 


o*  =  h* 

#4 

.961 

.790 

1.0 

(J 

#5 

.794 

1.0 

1.0 

#6 

.847 

.826 

.972 

Slack 

Values 

■ 

VEH 

#4 

2 

* 

* 

#5 

* 

* 

* 

#0 

* 

1 

* 

OGL 

#4 

3422 

* 

* 

#5 

15320 

* 

* 

#b 

* 

* 

* 

LAB  HR 

#4 

* 

* 

* 

#5 

* 

* 

* 

#6 

★ 

* 

* 

CWO 

#4 

* 

* 

* 

#5 

4 

* 

* 

#b 

* 

* 

19 

CJU 

#4 

28 

* 

* 

#5 

★ 

* 

* 

#6 

25 

175 

* 

LRWA 

#4 

* 

* 

* 

#5 

* 

★ 

* 

#b 

* 

* 

* 

Bjo 

*4 

5.2 

* 

* 

#5 

* 

* 

* 

#b 

* 

* 

* 

80 


NOTES 


1.  It  is  important  not  to  confuse  efficiency  with  effectiveness 
or  propriety.  Effectiveness  is  the  ability  to  state  and  achieve 
objectives.  Propriety  issues  ar  concerned  with  whether 
objectives  are  legal,  moral,  or  ethical  and  even  when  this  is  the 
case,  propriety  issues  can  arise  around  the  methods  used  to 
attain  the  objectives.  See  Kohler’s  Dictionary  For  Accountants 
[8]  p.  190  &  408. 

2.  This  term  is  used  to  indicate  a  not-for-profit  entity  in 
which  a  manager  has  some  freedom  of  decision  making  on  its  inputs 
and  outputs. 

3.  These  are  referred  to  as  "virtual  multipliers"  or 
"transformation  rates"  by  Charnes,  Cooper,  and  Rhodes  [6]. 

4.  Thus  h*  <  1  for  both  of  these  DMUs. 

5.  The  function  d(O-D’)  are  to  be  understood  as  measures  of 
distance  from  the  origin  in  a  Euclidean  metric. 

5a.  This  technique  was  used  in  Charnes,  Clark,  Cooper,  and  Golany 
t 3]  after  having  been  introduced  in  Charnes,  Cooper,  Divine, 
Klopp,  and  Stutz  [5]. 

5b.  bee  Charnes,  Cooper,  and  Rhodes  [6]  p.  433  for  a  proof. 

6.  Complete  results  for  the  window  analyses,  including  the  slack 
variable  values,  are  contained  in  Table  A. 3  in  Appendex  A. 

7.  In  the  Air  Force,  real  property  maintenance  supplies  are 
expensed  to  the  civil  engineering  account  when  they  are  received 
by  the  civil  engineer  and  not  when  they  are  actually  used  or 
consumed . 

8.  Refer  to  Table  A. 1  and  A. 2  in  Appendix  A  for  the  observed 
input  and  output  values  in  this  joint  analysis. 

9.  bee  the  results  of  the  anual  analysis  shown  in  Table  1  and 
the  window  analyses  reported  in  Table  2. 

10.  Table  A. 4  reporting  all  efficiency  measures  (0*  and  slack 
variables)  for  all  of  the  bases  is  located  in  Appendix  A. 

a 

M.  1'he  mein  was  calculated  by  averaging  the  average  ratin  (h„) 
for  each  quarter.  bo  for  Vance  we  have  the  following  average0 
rati ng  fur  each  quarter:  1st  -  .855,  2nd  -  .942,  3rd  -  .894,  and 
*th  -  These  averages  were  determined  from  the  h£  values 

*#>j  pi  lable  4.  The  average  of  these  averages  is  .91. 

#*.  ..  Mai  Analysis  #2  consisted  of  data  from  the  second, 

*  -urth  quarters  of  the  fiscal  year. 


i 
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13.  These  figures  were  calculated  by  dividing  the  completed  work 
type  from  Table  A.1  by  the  available  work  type  presented  in  Table 
*t.  tor  example,  Keesler's  second  quarter  measure  of  the 
percentage  of  completed  work  orders  was  computed  as  follows: 
81/925  =  .0876  =  8.76*. 
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